Robot action control method and device

Abstract

The invention provides a robot action control method and device, a robot, a storage medium and electronic equipment, and the method comprises the steps: obtaining first point cloud data in a target space including an activity area of a mechanical arm, then determining second point cloud data representing the mechanical arm, and removing the second point cloud data from the first point cloud data to obtain third point cloud data, and whether an obstacle exists or not is judged based on the third point cloud data to obtain a recognition result, and actions of the mechanical arm and / or the machine body are indicated and controlled based on the recognition result. Thus, due to the fact that interference of the point cloud data of the mechanical arm is filtered out of the third point cloud data, the accuracy of obstacle recognition can be improved, the obstacle avoidance requirement of the mechanical arm in the loading process is better met, the possibility that the robot collides with or makes contact with the obstacle to be damaged is reduced, and the safety of the robot is improved.

Description

Technical Field

[0001] This application relates to, but is not limited to, the field of robotics, and in particular, to a method and device for controlling the movement of a robot, a robot, a storage medium, and an electronic device. Background Art

[0002] Traditional robots mainly rely on infrared sensors, ultrasonic sensors, etc. for obstacle avoidance and navigation. However, in complex environments (e.g., areas with dense furniture, dynamic obstacle scenarios), the obstacle avoidance of traditional robots has limitations, and there are problems such as low safety and poor flexibility. Summary of the Invention

[0003] In view of this, embodiments of this application at least provide a method and device for controlling the movement of a robot, a robot, a storage medium, and an electronic device.

[0004] The technical solution of the embodiments of this application is implemented as follows:

[0005] Embodiments of this application provide a method for controlling the movement of a robot. The robot includes a fuselage and a robotic arm, and the robotic arm is arranged on the fuselage. The method includes:

[0006] Obtain first point cloud data of a target space; the target space includes the activity area of the robotic arm;

[0007] Determine second point cloud data representing the robotic arm, and remove the second point cloud data from the first point cloud data to obtain third point cloud data;

[0008] Based on the third point cloud data, determine whether there is an obstacle to obtain an identification result;

[0009] Based on the identification result, control the movement of the robotic arm and / or the fuselage.

[0010] In some embodiments, the robot further includes an acquisition component arranged on the fuselage; obtaining first point cloud data of the target space includes: controlling the acquisition component to acquire first point cloud data of the target space.

[0011] In some embodiments, based on the third point cloud data, determine whether there is an obstacle to obtain an identification result, including: determining that the third point cloud data meets the identification condition; wherein, the identification condition includes that the proportion of the point cloud data in the third point cloud data whose distance from the acquisition component is less than a distance threshold is greater than or equal to a preset ratio; based on the third point cloud data, determine whether there is an obstacle to obtain an identification result.

[0012] In some embodiments, determining the second point cloud data representing the robotic arm based on the current pose information of the robotic arm and the load state of the robotic arm includes: when the load state of the robotic arm represents that the robotic arm grasps the target obstacle, determining the second point cloud data based on the fourth point cloud data corresponding to the current pose information of the robotic arm and the fifth point cloud data corresponding to the current pose information of the target obstacle; or, when the load state of the robotic arm represents that the robotic arm does not grasp the target obstacle, determining the second point cloud data based on the fourth point cloud data.

[0013] In some embodiments, determining whether there is an obstacle based on the third point cloud data to obtain an identification result includes: performing clustering processing on the third point cloud data to obtain at least one first point cloud cluster; determining at least one target point cloud cluster from each of the first point cloud clusters based on the geometric feature information corresponding to the at least one first point cloud cluster; and determining the target object corresponding to each target point cloud cluster as the target obstacle.

[0014] In some embodiments, determining at least one target point cloud cluster from each of the first point cloud clusters based on the geometric feature information corresponding to the at least one first point cloud cluster includes: removing the second point cloud cluster representing the ground from the at least one first point cloud cluster to obtain at least one third point cloud cluster; the geometric feature information corresponding to the second point cloud cluster satisfies a first condition; the first condition includes that the surface flatness is greater than or equal to a first flatness threshold and the height is less than or equal to a height threshold; and determining at least one target point cloud cluster from each of the at least one third point cloud clusters based on the geometric feature information corresponding to the at least one third point cloud cluster.

[0015] In some embodiments, determining at least one target point cloud cluster from each of the first point cloud clusters based on the geometric feature information corresponding to the at least one first point cloud cluster includes: removing the fourth point cloud cluster representing the target grippable object from the at least one first point cloud cluster to obtain at least one fifth point cloud cluster; the geometric feature information corresponding to the fifth point cloud cluster satisfies a second condition; the second condition includes that the surface flatness is greater than or equal to a second flatness threshold, the volume is less than or equal to a first volume threshold, and the aspect ratio is less than or equal to a first aspect ratio threshold; and determining at least one target point cloud cluster from each of the at least one fifth point cloud clusters based on the geometric feature information corresponding to the at least one fifth point cloud cluster.

[0016] In some embodiments, the target obstacle includes a dynamic obstacle; the target point cloud cluster includes a first target point cloud cluster corresponding to the dynamic obstacle; determining at least one target point cloud cluster from each of the first point cloud clusters based on the geometric feature information corresponding to the at least one first point cloud cluster includes: when the geometric feature information corresponding to the first point cloud cluster satisfies a third condition, determining the first point cloud cluster as the first target point cloud cluster; the third condition includes that the surface flatness is less than or equal to a third flatness threshold and the displacement speed within a continuous plurality of time frames is greater than or equal to a displacement speed threshold.

[0017] In some embodiments, the target obstacle includes a static obstacle, and the target point cloud cluster includes a second target point cloud cluster corresponding to the static obstacle; based on geometric feature information corresponding to at least one first point cloud cluster, at least one target point cloud cluster is determined from each first point cloud cluster, including: when the geometric feature information corresponding to the first point cloud cluster satisfies a fourth condition, the first point cloud cluster is determined as the second target point cloud cluster; the fourth condition includes that the surface flatness is greater than or equal to a fourth flatness threshold and the volume is greater than or equal to a second volume threshold.

[0018] In some embodiments, based on the recognition results, the movement of the robotic arm and / or the fuselage is controlled, including: performing at least one of the following operations: controlling the fuselage and / or the robotic arm to stop moving; replanning the moving path, the moving path including at least one of the following: a first moving path of the robotic arm, a second moving path of the fuselage; outputting an alarm message.

[0019] In some embodiments, the replanning of the moving path is used to attempt to bypass the target obstacle, and includes: if the replanning is successful, controlling the robotic arm and / or the fuselage to move according to the replanned moving path; if the replanning fails, controlling the fuselage and / or the robotic arm to stop moving, and / or outputting an alarm message.

[0020] In some embodiments, when the target obstacle is a dynamic obstacle, before performing the operation, the step further includes: determining a safety status of the robotic arm, and performing at least one of the operations when the safety status of the robotic arm indicates that there is a safety risk to the robotic arm.

[0021] The present application embodiment provides a motion control device for a robot, the robot comprising a body and a mechanical arm, the mechanical arm being arranged on the body, the device comprising:

[0022] An acquisition module, used for acquiring first point cloud data of a target space; the target space includes an active area of ​​the robot arm;

[0023] A processing module, used for determining second point cloud data representing the robot arm, and removing the second point cloud data from the first point cloud data to obtain third point cloud data;

[0024] A recognition module, used to determine whether there is an obstacle based on the third point cloud data and obtain a recognition result;

[0025] The control module is used to control the movement of the robot arm and / or the body based on the recognition result.

[0026] In some embodiments, the robot also includes a collection component disposed on the body; the collection module is also used to control the collection component to collect the first point cloud data of the target space.

[0027] In some embodiments, the recognition module is further configured to determine that the third point cloud data meets the recognition condition; wherein, the recognition condition includes that the proportion of the point cloud data with a distance less than the distance threshold between the third point cloud data and the acquisition component in the third point cloud data is greater than or equal to a preset ratio; and based on the third point cloud data, determine whether there is an obstacle to obtain a recognition result.

[0028] In some embodiments, the processing module is further configured to, when the load state of the robotic arm represents that the robotic arm grasps a target obstacle, determine the second point cloud data based on the fourth point cloud data corresponding to the current pose information of the robotic arm and the fifth point cloud data corresponding to the current pose information of the target obstacle; or, when the load state of the robotic arm represents that the robotic arm does not grasp the target obstacle, determine the second point cloud data based on the fourth point cloud data.

[0029] In some embodiments, the recognition module is further configured to perform clustering processing on the third point cloud data to obtain at least one first point cloud cluster; based on the geometric feature information corresponding to the at least one first point cloud cluster, determine at least one target point cloud cluster from each of the first point cloud clusters; and determine the target objects corresponding to the target point cloud clusters as target obstacles.

[0030] In some embodiments, the recognition module is further configured to remove the second point cloud cluster representing the ground from the at least one first point cloud cluster to obtain at least one third point cloud cluster; the geometric feature information corresponding to the second point cloud cluster satisfies a first condition; the first condition includes that the surface flatness is greater than or equal to a first flatness threshold and the height is less than or equal to a height threshold; and based on the geometric feature information corresponding to the at least one third point cloud cluster, determine at least one target point cloud cluster from each of the third point cloud clusters.

[0031] In some embodiments, the recognition module is further configured to remove the fourth point cloud cluster representing the target graspable object from the at least one first point cloud cluster to obtain at least one fifth point cloud cluster; the geometric feature information corresponding to the fifth point cloud cluster satisfies a second condition; the second condition includes: the surface flatness is greater than or equal to a second flatness threshold, the volume is less than or equal to a first volume threshold, and the aspect ratio is less than or equal to a first aspect ratio threshold; and based on the geometric feature information corresponding to the at least one fifth point cloud cluster, determine at least one target point cloud cluster from each of the fifth point cloud clusters.

[0032] In some embodiments, the target obstacle includes a dynamic obstacle; the target point cloud cluster includes a first target point cloud cluster corresponding to the dynamic obstacle; the recognition module is further configured to, when the geometric feature information corresponding to the first point cloud cluster satisfies a third condition, determine the first point cloud cluster as the first target point cloud cluster; the third condition includes that the surface flatness is less than or equal to a third flatness threshold and the displacement speed within a continuous plurality of time frames is greater than or equal to a displacement speed threshold. In some embodiments,

[0033] In some embodiments, the target obstacle includes a static obstacle, and the target point cloud cluster includes a second target point cloud cluster corresponding to the static obstacle; the recognition module is further configured to determine the first point cloud cluster as the second target point cloud cluster when the geometric feature information corresponding to the first point cloud cluster meets the fourth condition; the fourth condition includes that the surface flatness is greater than or equal to a fourth flatness threshold and the volume is greater than or equal to a second volume threshold.

[0034] In some embodiments, the control module is further configured to perform at least one of the following operations: control the fuselage and / or the robotic arm to stop moving; re-plan the movement path, where the movement path includes at least one of the following: the first movement path of the robotic arm, the second movement path of the fuselage; output an alarm message.

[0035] In some embodiments, the control module is further configured to re-plan the movement path to attempt to bypass the target obstacle, and further includes: when the re-planning is successful, control the robotic arm and / or the fuselage to act according to the re-planned movement path; when the re-planning fails, control the fuselage and / or the robotic arm to stop moving, and / or output an alarm message.

[0036] In some embodiments, when the target obstacle is a dynamic obstacle, before performing the operation, the control module is further configured to determine the safety state of the robotic arm, and when the safety state of the robotic arm indicates that there is a safety risk for the robotic arm, perform at least one of the operations.

[0037] An embodiment of the present application provides a robot, including a processor and a memory, where the memory stores a computer program that can run on the processor, and the processor implements the steps in the above-mentioned motion control method of the robot when executing the computer program.

[0038] An embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps in the above-mentioned motion control method of the robot are implemented.

[0039] An embodiment of the present application provides a computer program product, characterized by including a computer program or instruction, and when the computer program or instruction is executed by a processor, the steps in the above-mentioned motion control method of the robot are implemented.

[0040] In the embodiments of the present application, first point cloud data in a target space including the active area of the robotic arm is obtained, then second point cloud data representing the robotic arm is determined, and the second point cloud data is removed from the first point cloud data to obtain third point cloud data. Furthermore, based on the third point cloud data, it is determined whether there is an obstacle to obtain an identification result, so as to control the actions of the robotic arm and / or the fuselage based on the identification result. In this way, since the interference of the point cloud data of the robotic arm is filtered out in the third point cloud data, the accuracy of obstacle identification can be improved, thus better meeting the obstacle avoidance requirements of the robotic arm during the loading process, reducing the possibility of damage caused by the robot colliding with or contacting an obstacle, and improving the safety of the robot.

[0041] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and do not limit the technical solutions of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The accompanying drawings herein are incorporated into the specification and constitute a part of this specification. These drawings illustrate embodiments consistent with the present application and, together with the specification, are used to explain the technical solutions of the present application.

[0043] Figure 1 Schematic diagram of the implementation process of a method for controlling the actions of a robot provided by an embodiment of the present application Figure 1 ;

[0044] Figure 2 Schematic diagram of the implementation process of a method for controlling the actions of a robot provided by an embodiment of the present application Figure 2 ;

[0045] Figure 3 Schematic diagram of the composition structure of a device for controlling the actions of a robot provided by an embodiment of the present application;

[0046] Figure 4 Schematic diagram of the composition structure of a robot provided by an embodiment of the present application Figure 1 ;

[0047] Figure 5 Schematic diagram of the composition structure of a robot provided by an embodiment of the present application Figure 2 ;

[0048] Figure 6 Schematic diagram of the composition structure of a robot provided by an embodiment of the present application Figure 3 。 DETAILED DESCRIPTION OF THE EMBODIMENTS

[0049] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be further elaborated in detail below with reference to the accompanying drawings and embodiments. The described embodiments should not be construed as limitations on this application. All other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the scope of protection of this application.

[0050] In the following description, reference is made to "some embodiments" which describe a subset of all possible embodiments. However, it can be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict.

[0051] It should be noted that the terms "first / second / third" involved in the embodiments of this application are only used to distinguish similar objects and do not represent a specific order for the objects. It can be understood that "first / second / third" can be interchanged with a specific order or sequence when permitted, so that the embodiments of this application described here can be implemented in an order other than that illustrated or described here.

[0052] Those skilled in the art of this technology can understand that, unless otherwise defined, all terms used here (including technical terms and scientific terms) have the same meaning as the general understanding of those of ordinary skill in the art in the field to which the embodiments of this application belong. It should also be understood that terms such as those defined in a general dictionary should be understood to have a meaning consistent with the meaning in the context of the prior art, and will not be interpreted with an idealized or overly formal meaning unless specifically defined as here.

[0053] With the rapid development of smart home technology, robots have become an important tool for modern household cleaning. Traditional robots usually use infrared sensors, ultrasonic sensors, etc. to achieve obstacle avoidance and navigation. In complex environments (such as areas with dense furniture, dynamic obstacle scenarios), the obstacle avoidance of traditional robots has limitations. For example, infrared sensors are easily interfered by ambient light, and ultrasonic sensors have large ranging errors in complex reflection scenarios. At the same time, traditional robots have a long data processing cycle and are difficult to meet the real-time obstacle avoidance requirements of robotic arms, resulting in problems such as low safety and poor flexibility.

[0054] The embodiments of this application provide a method for controlling the actions of a robot, which can be applied to a robot. Among them, the robot can be a robot with a fixed position or a mobile robot; a mobile robot refers to a robot that can move autonomously. For example, the robot can include, but is not limited to, at least one of cleaning robots (such as sweepers, floor washers, mopping machines, scrubbing and mopping all-in-one machines, etc.), guiding robots, service robots, etc. Figure 1 Schematic implementation process of a method for controlling the actions of a robot provided by the embodiments of this application Figure 1, the robot includes a fuselage and a robotic arm, and the robotic arm is arranged on the fuselage, as Figure 1 shown, the method includes the following steps S101 to step S104:

[0055] Step S101, obtaining first point cloud data of the target space; the target space includes the activity area of the robotic arm.

[0056] Here, the point cloud data (including the first point cloud data, the second point cloud data, and the point cloud data mentioned later) refers to a set of points expressing the distribution of the target space and the characteristics of the target surface in the same three-dimensional coordinate system.

[0057] In some embodiments, the robotic arm refers to a complex system with high precision, multiple inputs and outputs, high nonlinearity, and strong coupling. In some embodiments, the robotic arm is arranged on the top of the fuselage, and the robotic arm may include at least one joint and an end effector. The robotic arm can grasp an object through the end effector.

[0058] In some embodiments, the fuselage includes a control component and a chassis drive module. The control component has a control function and can issue control instructions to the components of the robot to control the robot. The chassis drive module can drive the fuselage to move. In implementation, the control component may include, but is not limited to, one of the following: Central Processing Unit (CPU), Microcontroller Unit (MCU), Single Chip Microcomputer (SCM), etc. The CPU is the operation and control core of the computer system and the final execution unit for information processing and program operation. The MCU appropriately reduces the frequency and specifications of the CPU and integrates peripheral interfaces such as memory and counters on a single chip to form a chip-level computer, which can perform different combined controls for different application scenarios. The SCM is a microcomputer that integrates the main computer functional components such as the CPU, Random Access Memory (RAM), Read-Only Memory (ROM), and input / output ports on an integrated circuit chip.

[0059] In some embodiments, the robotic arm is installed in a receiving bin on the top of the fuselage, and the robotic arm can perform operations including grasping obstacles when receiving control instructions sent by the control component.

[0060] The target space refers to the activity space of the robot. The target space may include the activity area of ​​the robot. The activity area of ​​the robot refers to the set of all positions that the robot can reach in space. In some embodiments, the activity area of ​​the robot is related to the structural parameters of the robot. The structural parameters of the robot may include but are not limited to at least one of the following: the joint type of the robot, the connecting rod length of the robot, the joint angle limit of the robot, etc.

[0061] In some embodiments, the robot further includes an upper server that supports a communication protocol. The upper server can interact with the control component through a variety of networks or wireless protocols so that the control component can implement a control function.

[0062] In some embodiments, the robot further includes a collection component disposed at the front portion of the body, and the collection component can be controlled to collect first point cloud data of the target space.

[0063] Step S102, determining second point cloud data representing the robotic arm, and removing the second point cloud data from the first point cloud data to obtain third point cloud data.

[0064] Here, the third point cloud data refers to the point cloud data obtained after removing the point cloud data representing the robot arm. In some embodiments, the number of point cloud data in the second point cloud data is less than the number of point cloud data in the first point cloud data. The number of point cloud data in the third point cloud data is less than the number of point cloud data in the first point cloud data.

[0065] In some embodiments, data may be extracted from the first point cloud data to reconstruct a point cloud model of the robotic arm, and the point cloud data included in the point cloud model of the robotic arm is the second point cloud data characterizing the robotic arm.

[0066] In some embodiments, the point cloud space where the robotic arm is located can be determined based on the current posture information of the robotic arm and the load status of the robotic arm, and the point cloud data in the point cloud space (i.e., the second point cloud data characterizing the robotic arm) can be removed from the first point cloud data to obtain third point cloud data.

[0067] The methods for determining the third point cloud data may include, but are not limited to: obtaining the third point cloud data by removing the second point cloud data representing the robotic arm from the first point cloud data through a point cloud segmentation algorithm, obtaining the third point cloud data by removing the second point cloud data representing the robotic arm from the first point cloud data through coordinate transformation, and the like. For example, the point cloud segmentation algorithm may include, but is not limited to, at least one of the following: region growing segmentation algorithm, model-based segmentation algorithm, etc. The point cloud segmentation algorithm can be used to remove the second point cloud data representing the robotic arm from the first point cloud data to obtain the third point cloud data. Also, for example, the collected first point cloud data can be transformed from the sensor coordinate system to the base coordinate system of the robotic arm, and then the second point cloud data representing the robotic arm can be removed from the first point cloud data to obtain the third point cloud data.

[0068] In some embodiments, when the load state of the robotic arm represents that the robotic arm grasps the target obstacle, the second point cloud data can be determined based on the fourth point cloud data corresponding to the current pose information of the robotic arm and the fifth point cloud data corresponding to the current pose information of the target obstacle. When the load state of the robotic arm represents that the robotic arm does not grasp the target obstacle, the second point cloud data can be determined based on the fourth point cloud data.

[0069] Step S103, determining whether there is an obstacle based on the third point cloud data to obtain an identification result.

[0070] Here, the identification result can represent whether there is a target obstacle in the target space.

[0071] In some embodiments, the third point cloud data can be preprocessed to obtain the preprocessed third point cloud data, and then it is determined whether there is an obstacle based on the preprocessed third point cloud data to obtain an identification result, so as to improve the accuracy of determining the identification result while reducing the computing amount of the robot. The preprocessing of the point cloud data may include, but is not limited to, at least one of the following: filtering processing, coordinate calibration, coordinate transformation, etc.

[0072] In some embodiments, the geometric features corresponding to the third point cloud data can be extracted from the third point cloud data, and then it is determined whether there is an obstacle based on the geometric features corresponding to the third point cloud data to obtain an identification result. The geometric features corresponding to the point cloud data may include, but are not limited to, one of the following: normal information, curvature information, etc. For example, if there is a changing curvature information in the geometric features corresponding to the point cloud data, it can be used as the identification result that there is a target obstacle in the target space.

[0073] In some embodiments, the third point cloud data can be voxelized, and then a convolutional neural network can be used to determine whether there are obstacles to obtain an identification result. Voxelization refers to the process of dividing point cloud data into small cubes to extract point cloud features. For example, VoxelNet is an end-to-end object detection framework that can first voxelize the third point cloud data through a voxel feature encoder, and then use the voxelized third point cloud data to determine whether there are obstacles to obtain an identification result.

[0074] In some embodiments, the third point cloud data can be first subjected to height filtering processing, and then based on the height-filtered third point cloud data, it can be determined whether there are obstacles to obtain an identification result. Height filtering is a technique for processing height data in point cloud data, which can eliminate noise in the point cloud data and improve the accuracy of height identification. In implementation, height filtering processing can be performed by methods including but not limited to Kalman filtering, Gaussian filtering, etc., and the present application does not limit this.

[0075] In some embodiments, a first correspondence relationship can be established between the point cloud data and the identification result, so that after the third point cloud data is determined, it can be determined whether there are obstacles based on the first correspondence relationship and the third point cloud data to obtain an identification result.

[0076] In some embodiments, it can be determined whether the third point cloud data meets the identification conditions. When the third point cloud data meets the identification conditions, it is determined whether there are obstacles based on the third point cloud data to obtain an identification result; wherein, the identification conditions include that the proportion of the point cloud data with a distance less than the distance threshold from the acquisition component in the third point cloud data is greater than or equal to a preset ratio.

[0077] In some embodiments, the third point cloud data can be clustered to obtain at least one first point cloud cluster, and then based on the geometric feature information corresponding to the at least one first point cloud cluster, at least one target point cloud cluster can be determined from each first point cloud cluster, so as to determine the target object corresponding to each target point cloud cluster as a target obstacle.

[0078] Step S104, based on the identification result, control the actions of the robotic arm and / or the fuselage.

[0079] Here, the actions of the robotic arm and / or the fuselage can be controlled based on the identification result for obstacle avoidance processing, so that the robot can bypass or avoid obstacles.

[0080] In some embodiments, when at least one target obstacle is included in the recognition result, the recognition result may indicate the presence of an obstacle, and the actions of the robotic arm and / or the fuselage may be controlled to perform obstacle avoidance processing. When the recognition result does not include the target obstacle, the first point cloud data of the target space may be acquired again and the recognition result may be determined.

[0081] In some embodiments, controlling the actions of the robotic arm may include, but is not limited to, at least one of the following: adjusting the movement path of the robotic arm, adjusting the joint angles of the robotic arm, adjusting the telescopic length of the robotic arm, controlling the robotic arm to stop moving, etc. The actions performed by controlling the fuselage may include, but are not limited to, at least one of the following: adjusting the movement direction of the fuselage, adjusting the movement speed of the fuselage, controlling the fuselage to stop moving, etc.

[0082] In some embodiments, after determining the recognition result, the components that need to control the actions may be determined. When it is necessary to control the actions of the robotic arm and the fuselage, the actions of the robotic arm and the fuselage are controlled. When it is necessary to control the action of the robotic arm or the fuselage, the action of the robotic arm or the fuselage is controlled.

[0083] In the embodiments of the present application, the first point cloud data in the target space including the activity area of the robotic arm is acquired, then the second point cloud data representing the robotic arm is determined, and the second point cloud data is removed from the first point cloud data to obtain the third point cloud data. Furthermore, based on the third point cloud data, it is determined whether there is an obstacle to obtain the recognition result, so as to control the actions of the robotic arm and / or the fuselage based on the recognition result. In this way, since the interference of the point cloud data of the robotic arm is filtered out in the third point cloud data, the accuracy of obstacle recognition can be improved, thus better meeting the obstacle avoidance requirements of the robotic arm during the loading process, reducing the possibility of damage caused by the robot colliding with or contacting an obstacle, and improving the safety of the robot.

[0084] In some embodiments, the robot further includes an acquisition component disposed on the fuselage; "acquiring the first point cloud data of the target space" in the above step S101 may include the following step S111:

[0085] Step S111, controlling the acquisition component to acquire the first point cloud data of the target space.

[0086] Here, the acquisition component is a component having the functions of acquiring data, recording data, and transmitting data.

[0087] In some embodiments, the acquisition component may include but is not limited to at least one of the following: at least one of a depth camera, a time of flight (TOF) sensor, etc. A TOF sensor is a depth measurement sensor that uses the flight time of a light pulse to measure the distance of an object. In some embodiments, the number of acquisition components is at least one, and at least one acquisition component is disposed at the front of the fuselage. In some embodiments, the acquisition component may be disposed at the front, side, and / or top of the fuselage.

[0088] In some embodiments, the acquisition component can be set at the front of the fuselage, so that the acquisition component can cover the field of view forward and obtain the first point cloud data in front of the fuselage, thereby detecting the target obstacle in front of the fuselage. In implementation, the acquisition component can be set at any suitable position in the front of the fuselage, and the embodiment of the present application is not limited to this. For example, the acquisition component can be set at the left position of the front of the fuselage, the right position of the front of the fuselage, or the center position of the front of the fuselage.

[0089] In the above embodiment, when the robot arm is performing the operation of grabbing obstacles, the activity area of ​​the robot arm is in the space near the fuselage, and the acquisition component arranged on the fuselage can accurately acquire the point cloud data of the target space, thereby improving the accuracy of obstacle recognition.

[0090] In some embodiments, the above step S103 may include the following steps S131 and S132:

[0091] Step S131, determining that the third point cloud data meets the recognition condition; wherein the recognition condition includes that the proportion of point cloud data in the third point cloud data whose distance to the acquisition component is less than a distance threshold is greater than or equal to a preset ratio.

[0092] Here, the recognition condition may be a condition for determining whether a recognition result is obtained.

[0093] In some embodiments, the identification condition includes that the proportion of point cloud data in the third point cloud data whose distance from the acquisition component is less than the distance threshold is greater than or equal to a preset ratio. The distance between the third point cloud data and the acquisition component can be any suitable size, for example, 13m (meters), 25cm (centimeter), etc. The distance threshold can be any suitable size, for example, 10cm, 15cm, etc. The preset ratio can be any suitable size, for example, 60%, 0.5, etc. The proportion of point cloud data in the third point cloud data whose distance from the acquisition component is less than the distance threshold can be any suitable size, for example, 10%, 0.22, etc.

[0094] In some embodiments, the method for determining the distance between the third point cloud data and the acquisition component may include, but is not limited to: determining the distance between the point cloud data in the third point cloud data and the acquisition component through the Euclidean distance formula, determining the distance between the point cloud data in the third point cloud data and the acquisition component through computational geometry methods (such as Voronoi diagrams, Delaunay triangulations, etc.), determining the distance between the third point cloud data and the acquisition component through numerical methods (such as gradient descent method, simulated annealing method, etc.). Among them, computational geometry methods may include at least one of the following: Voronoi diagrams, Delaunay triangulations, etc. Numerical methods may include at least one of the following: gradient descent method, simulated annealing method, etc.

[0095] In some embodiments, the third point cloud data may be continuously acquired, and then the point cloud data in the third point cloud data with a distance less than the distance threshold from the acquisition component may be continuously screened.

[0096] In some embodiments, when it is determined that the third point cloud data meets the recognition conditions, it may be further determined whether there is an obstacle based on the third point cloud data to obtain a recognition result. When it is determined that the third point cloud data does not meet the recognition conditions, the third point cloud data may be acquired again and it may be determined whether the third point cloud data meets the recognition conditions until the third point cloud data that meets the recognition conditions is obtained.

[0097] In some embodiments, when the proportion of the point cloud data in the third point cloud data with a distance less than the distance threshold from the acquisition component is less than the preset ratio, the point cloud data in the third point cloud data with a distance less than the distance threshold may be screened again until the proportion of the point cloud data in the third point cloud data with a distance less than the distance threshold in the third point cloud data is greater than or equal to the preset ratio, and then it may be determined whether there is an obstacle based on the third point cloud data to obtain a recognition result.

[0098] Step S132: Determine whether there is an obstacle based on the third point cloud data to obtain a recognition result.

[0099] Here, the specific implementation process of step S132 may refer to the specific implementation manner of the foregoing step S103.

[0100] In the above embodiments, only when the third point cloud data meets the recognition conditions, it will be determined whether there is an obstacle based on the third point cloud data to obtain a recognition result, which can reduce the consumption of computing resources caused by continuously determining the recognition result and improve the flexibility of the robot's motion control method.

[0101] In some embodiments, "determining the second point cloud data representing the robotic arm" in the above step S102 may include the following step S121:

[0102] Step S121: Based on the current pose information of the robotic arm and the load state of the robotic arm, determine the second point cloud data representing the robotic arm.

[0103] Here, the current pose information of the robotic arm refers to the position and orientation information of the robotic arm in space. In some embodiments, the current pose information of the robotic arm can be represented by a six-dimensional vector, where the first three dimensions are position information and the last three dimensions are orientation information.

[0104] In some embodiments, the load state of the robotic arm can be used to represent whether the robotic arm grasps an obstacle. The load state of the robotic arm can represent that the robotic arm grasps the target obstacle or that the robotic arm does not grasp the target obstacle.

[0105] In some embodiments, when the load state of the robotic arm represents that the robotic arm grasps the target obstacle, the second point cloud data can be determined based on the fourth point cloud data corresponding to the current pose information of the robotic arm and the fifth point cloud data corresponding to the current pose information of the target obstacle. When the load state of the robotic arm represents that the robotic arm does not grasp the target obstacle, the second point cloud data can be determined based on the fourth point cloud data.

[0106] In the above embodiments, since the contour information of the robotic arm may be different under different load states, by combining the current pose information of the robotic arm and the load state of the robotic arm, the point cloud data representing the robotic arm can be determined more accurately, so that the possibility of misjudging obstacles can be reduced during the process of obstacle recognition.

[0107] In some embodiments, the "Based on the current pose information of the robotic arm and the load state of the robotic arm, determine the second point cloud data representing the robotic arm" in the above step S121 may include at least one of the following steps S1211 and step S1212:

[0108] Step S1211: When the load state of the robotic arm represents that the robotic arm grasps the target obstacle, determine the second point cloud data based on the fourth point cloud data corresponding to the current pose information of the robotic arm and the fifth point cloud data corresponding to the current pose information of the target obstacle.

[0109] Here, the second point cloud data refers to the point cloud data representing the robotic arm. The second point cloud data may include the point cloud data of the robotic arm and the point cloud data of the obstacle.

[0110] In some embodiments, when the robotic arm grasps an obstacle, the robotic arm and the obstacle can be regarded as a whole. Therefore, the second point cloud data is determined based on the fourth point cloud data corresponding to the current pose information of the robotic arm and the fifth point cloud data corresponding to the current pose information of the target obstacle.

[0111] In some embodiments, the fourth point cloud data refers to the point cloud data corresponding to the current pose information of the robotic arm. The fifth point cloud data refers to the point cloud data corresponding to the current pose information of the target obstacle. The target obstacle is an obstacle in the target space.

[0112] In some embodiments, the fourth point cloud data and the fifth point cloud data can be merged, and then the merged point cloud data is used as the second point cloud data to determine the second point cloud data. In implementation, the fourth point cloud data and the fifth point cloud data can be merged by vertical merging or horizontal merging.

[0113] In some embodiments, the merged point cloud data can be preprocessed, and then the preprocessed merged point cloud data is used as the second point cloud data to determine the second point cloud data. The point cloud data preprocessing can include but is not limited to at least one of the following: filtering processing, coordinate calibration, coordinate transformation, etc.

[0114] Step S1212, when the load state of the robotic arm indicates that the robotic arm has not grasped the target obstacle, based on the fourth point cloud data, determine the second point cloud data.

[0115] Here, when the robotic arm does not grasp the obstacle, it is not necessary to consider the point cloud data corresponding to the obstacle. Therefore, the second point cloud data is determined based on the fourth point cloud data corresponding to the current pose information of the robotic arm.

[0116] In some embodiments, the fourth point cloud data can be directly used as the second point cloud data to determine the second point cloud data.

[0117] In some embodiments, the fourth point cloud data can be preprocessed, and then the preprocessed fourth point cloud data is used as the second point cloud data to determine the second point cloud data. The point cloud data preprocessing can include but is not limited to at least one of the following: filtering processing, coordinate calibration, coordinate transformation, etc.

[0118] It should be noted that the focus of the solution of this application is on controlling the actions of the robot to achieve obstacle avoidance processing. Therefore, this application does not limit the specific process of the robotic arm to determine whether to grasp the obstacle. However, whether the robotic arm grasps the target obstacle will affect the process of determining the second point cloud data.

[0119] In the above embodiments, by different situations of the load state of the robotic arm, the second point cloud data is determined only based on the fourth point cloud data or based on the fourth point cloud data and the fifth point cloud data, which can flexibly determine the second point cloud data according to the actual usage scenario and improve the accuracy of the second point cloud data.

[0120] In some embodiments, the recognition result includes at least one target obstacle; the above step S103 may include the following steps S133 to S135:

[0121] Step S133: Perform clustering processing on the third point cloud data to obtain at least one first point cloud cluster.

[0122] Here, clustering processing is a statistical learning method that can group data objects with similar characteristics into the same class. A point cloud cluster is the point cloud data after clustering processing. The number of the first point cloud clusters is at least one.

[0123] The method for clustering the third point cloud data may include, but is not limited to: performing clustering processing on the third point cloud data through the K-Means clustering algorithm, performing clustering processing on the third point cloud data through the mean shift clustering algorithm, etc. For example, K-Means clustering can set at least one category and divide each third point cloud data into the cluster represented by the nearest cluster center in an iterative manner to obtain at least one first point cloud cluster. Another example is that mean shift clustering can determine a window radius and start sliding from a randomly selected center point. Each time it slides to a new area, calculate the mean value within the window as the center point. When multiple sliding windows overlap, stop sliding and obtain at least one center point. Finally, divide each third point cloud data based on the center point to obtain at least one first point cloud cluster.

[0124] Step S134: Based on the geometric feature information corresponding to at least one first point cloud cluster, determine at least one target point cloud cluster from each first point cloud cluster.

[0125] Here, the target point cloud cluster is the point cloud cluster corresponding to the target obstacle. The number of the target point cloud clusters is at least one.

[0126] In some embodiments, the geometric feature information corresponding to a point cloud cluster (including the first point cloud cluster, the third point cloud cluster, and the point cloud clusters mentioned later) may include, but is not limited to, at least one of the following: volume, aspect ratio, surface flatness, height, etc. Among them, the volume can be any appropriate size. For example, 0.1 m 3 (cubic meters), 3 m 3 etc. The aspect ratio can be any appropriate size. For example, 3:1, 1.2:3, etc. The surface flatness can be any appropriate size. For example, 70%, 0.4, etc. The height can be any appropriate size. For example, 0.1 m, 15 cm, etc.

[0127] In some embodiments, the surface flatness can be used to determine whether the shape of the obstacle is regular and / or whether the surface of the obstacle is continuous.

[0128] In some embodiments, a point cloud cluster can be converted into multiple three-dimensional mesh models, and then volume integration can be performed on each mesh model, and finally the volumes are accumulated to obtain the volume. Principal component analysis can be performed on the point cloud cluster to obtain the principal axis direction of the point cloud, and then the point cloud cluster can be projected onto the principal axis direction to calculate the length of the projection, and the aspect ratio can be determined based on the ratio of the longest axis to the second longest axis. The surface equation of the point cloud cluster can be fitted, and then the normal vector and principal curvature of the surface equation can be calculated to determine the surface flatness based on the normal vector and principal curvature.

[0129] In some embodiments, a second correspondence can be established between the geometric feature information and the point cloud cluster. After determining the geometric feature information corresponding to at least one first point cloud cluster, at least one target point cloud cluster can be determined from each of the first point cloud clusters based on this second correspondence.

[0130] In some embodiments, a preset rule library can be established. After determining the geometric feature information corresponding to at least one first point cloud cluster, all the point cloud clusters in the first point cloud cluster can be traversed based on this rule library, and the point cloud clusters that meet the preset rules can be screened out, so that at least one target point cloud cluster can be determined.

[0131] In some embodiments, the second point cloud clusters representing the ground in at least one first point cloud cluster can be removed to obtain at least one third point cloud cluster, and then at least one target point cloud cluster can be determined from each of the third point cloud clusters based on the geometric feature information corresponding to the at least one third point cloud cluster. Among them, the geometric feature information corresponding to the second point cloud cluster satisfies a first condition, and the first condition includes that the surface flatness is greater than or equal to a first flatness threshold and the height is less than or equal to a height threshold.

[0132] In some embodiments, the fourth point cloud clusters representing the target graspable objects in at least one first point cloud cluster can be removed to obtain at least one fifth point cloud cluster, and then at least one target point cloud cluster can be determined from each of the fifth point cloud clusters based on the geometric feature information corresponding to the at least one fifth point cloud cluster. Among them, the geometric feature information corresponding to the fifth point cloud cluster satisfies a second condition, and the second condition includes: the surface flatness is greater than or equal to a second flatness threshold, the volume is less than or equal to a first volume threshold, and the aspect ratio is less than or equal to a first aspect ratio threshold.

[0133] In some embodiments, the target obstacle includes a dynamic obstacle; the target point cloud cluster includes a first target point cloud cluster corresponding to the dynamic obstacle. The first point cloud cluster can be determined as the first target point cloud cluster when the geometric feature information corresponding to the first point cloud cluster satisfies a third condition. Among them, the third condition includes that the surface flatness is less than or equal to a third flatness threshold and the displacement speed within a continuous plurality of time frames is greater than or equal to a displacement speed threshold.

[0134] In some embodiments, the target obstacle includes a static obstacle, and the target point cloud cluster includes a second target point cloud cluster corresponding to the static obstacle. The first point cloud cluster can be determined as the second target point cloud cluster when the geometric feature information corresponding to the first point cloud cluster meets the fourth condition. The fourth condition includes that the surface flatness is greater than or equal to a fourth flatness threshold and the volume is greater than or equal to a second volume threshold.

[0135] Step S135: Determine the target object corresponding to each target point cloud cluster as the target obstacle.

[0136] Here, the target object refers to the object corresponding to the target point cloud cluster. The number of target obstacles is at least one.

[0137] In some embodiments, the target obstacle may include a static obstacle and / or a dynamic obstacle. When it is detected that the position of the target point cloud cluster changes within a continuous plurality of time frames, the target obstacle can be determined as a dynamic obstacle. When it is detected that the position of the target point cloud cluster remains unchanged within a continuous plurality of time frames, the target obstacle can be determined as a static obstacle.

[0138] In some embodiments, the target objects corresponding to different target point cloud clusters may be the same target object or different target objects. That is, the target obstacle can be represented by at least one target point cloud cluster.

[0139] In the above embodiments, the second point cloud data is clustered to obtain at least one point cloud cluster, and thus the target point cloud cluster is determined from each first point cloud cluster based on the geometric feature information corresponding to at least one first point cloud cluster, improving the accuracy of the determination result.

[0140] In some embodiments, the above step S134 may include the following steps S1341 and S1342:

[0141] Step S1341: Remove the second point cloud cluster representing the ground from at least one first point cloud cluster to obtain at least one third point cloud cluster; the geometric feature information corresponding to the second point cloud cluster meets the first condition; the first condition includes that the surface flatness is greater than or equal to a first flatness threshold and the height is less than or equal to a height threshold.

[0142] Here, the second point cloud cluster refers to the point cloud cluster representing the ground. The third point cloud cluster refers to the point cloud cluster obtained by removing the second point cloud cluster from the first point cloud cluster. In some embodiments, the number of second point cloud clusters is at least one, and the number of third point cloud clusters is at least one. The number of third point cloud clusters is less than or equal to the number of first point cloud clusters.

[0143] In some embodiments, the geometric feature information corresponding to the second point cloud cluster satisfies a first condition. The first condition includes that the surface flatness is greater than or equal to a first flatness threshold and the height is less than or equal to a height threshold. The first flatness threshold can be of any suitable size, for example, 60%, 0.8, etc. The height threshold can be of any suitable size, for example, 0.1 m, 8 cm, etc.

[0144] In some embodiments, the second point cloud cluster representing the ground can be removed from the first point cloud cluster in the first point cloud data through a point cloud segmentation algorithm, obtaining at least one third point cloud cluster. The point cloud segmentation algorithm can include, but is not limited to, at least one of the following: region growing segmentation algorithm, model-based segmentation algorithm, etc.

[0145] Exemplarily, the first condition includes that the surface flatness is greater than or equal to 80% and the height is less than or equal to 10 cm. The point cloud cluster in the first point cloud cluster whose geometric feature information satisfies the first condition can be used as the second point cloud cluster representing the ground, and the second point cloud cluster is removed from the first point cloud cluster to obtain at least one third point cloud cluster. Among them, the object corresponding to the point cloud cluster in the first point cloud cluster that satisfies the surface flatness greater than or equal to 80% and the height less than or equal to 10 cm is the ground. It can be understood that the ground is a regular continuous plane. By simultaneously restricting the surface flatness and the height, the second point cloud cluster representing the ground in the first point cloud cluster can be accurately identified. Removing the second point cloud cluster can reduce the interference of the ground, thereby improving the accuracy of the recognition result.

[0146] Step S1342, based on the geometric feature information corresponding to at least one third point cloud cluster, determine at least one target point cloud cluster from each of the third point cloud clusters.

[0147] Here, the number of target point cloud clusters is at least one. The geometric feature information corresponding to the geometric feature information of the third point cloud cluster can include, but is not limited to, at least one of the following: volume, aspect ratio, surface flatness, height, etc.

[0148] In some embodiments, a preset rule library can be established. After determining the geometric feature information corresponding to at least one third point cloud cluster, all the point cloud clusters in the third point cloud cluster can be traversed based on this rule library, and the point cloud clusters that satisfy the preset rules in the third point cloud cluster can be screened out, thereby determining at least one target point cloud cluster.

[0149] In the above embodiments, by screening the second point cloud cluster representing the ground through the first condition and removing the second point cloud cluster in at least one first point cloud cluster, the influence of the point cloud data of the ground can be reduced when identifying the target obstacle, and the accuracy of the recognition result is improved.

[0150] In some embodiments, the above step S134 may include the following steps S1343 and S1344:

[0151] Step S1343, removing the fourth point cloud clusters representing the target graspable objects from at least one first point cloud cluster to obtain at least one fifth point cloud cluster; the geometric feature information corresponding to the fifth point cloud cluster satisfies the second condition; the second condition includes: the surface flatness is greater than or equal to the second flatness threshold, the volume is less than or equal to the first volume threshold, and the aspect ratio is less than or equal to the first aspect ratio threshold.

[0152] Here, the fourth point cloud cluster refers to the point cloud cluster representing the graspable object. The fifth point cloud cluster refers to the point cloud cluster obtained by removing the fourth point cloud cluster from the first point cloud cluster. In some embodiments, the number of the fourth point cloud clusters is at least one, and the number of the fifth point cloud clusters is at least one. The number of the fifth point cloud clusters is less than or equal to the number of the first point cloud clusters.

[0153] In some embodiments, the geometric feature information corresponding to the fifth point cloud cluster satisfies the second condition. The second condition includes: the surface flatness is greater than or equal to the second flatness threshold, the volume is less than or equal to the first volume threshold, and the aspect ratio is less than or equal to the first aspect ratio threshold. That is, the point cloud cluster satisfying all the above conditions is the fifth point cloud cluster. Among them, the second flatness threshold, the first volume threshold, and the first aspect ratio threshold can all be determined according to the object grasping ability of the robotic arm.

[0154] In some embodiments, the second flatness threshold can be of any suitable size, for example, 70%, 0.68, etc. The functions of the second flatness threshold and the first flatness threshold are different, and their sizes can be the same or different. The first volume threshold is of any suitable size, for example, 0.1m 3 , 0.25m 3 etc. The first aspect ratio threshold can be of any suitable size, for example, 3:1, 2:1, etc.

[0155] In some embodiments, the second condition can be determined according to the object grasping ability of the robotic arm. The object grasping ability of the robotic arm can be used to characterize the geometric information corresponding to the objects that the robotic arm can grasp. The grasping ability of the robotic arm is related to the structural parameters of the robotic arm. The structural parameters of the robotic arm can include, but are not limited to, at least one of the following: the joint type of the robotic arm, the link length of the robotic arm, the joint angle limit of the robotic arm, etc.

[0156] Exemplarily, the second condition includes that the surface flatness is greater than or equal to 70%, the volume is less than or equal to 0.1m 3 , and the aspect ratio is less than or equal to 3:1. The point cloud clusters in the first point cloud cluster whose geometric feature information satisfies the second condition can be used as the fourth point cloud clusters representing the target graspable objects, and the fourth point cloud clusters are removed from the first point cloud cluster to obtain at least one third point cloud cluster. Among them, in the first point cloud cluster, those satisfying the surface flatness greater than or equal to 70%, the volume less than or equal to 0.1m 3An object corresponding to a point cloud cluster with an aspect ratio less than or equal to 3:1 is a target object that can be grasped. It can be understood that since the robotic arm is prone to dropping objects when gripping objects with a relatively low surface flatness (such as objects with irregular shapes), and the grasping ability of the robotic arm also affects the volume and / or aspect ratio of the target object that can be grasped. Therefore, through the joint limitation of surface flatness, volume, and aspect ratio, the fourth point cloud cluster representing the target object that can be grasped in the first point cloud cluster can be accurately identified, and removing the fourth point cloud cluster can reduce the interference of the target object that can be grasped, thereby improving the accuracy of the recognition result.

[0157] Step S1344, based on the geometric feature information corresponding to at least one fifth point cloud cluster, determine at least one target point cloud cluster from each of the fifth point cloud clusters.

[0158] Here, the number of target point cloud clusters is at least one. The geometric feature information corresponding to the fifth point cloud cluster may include, but is not limited to, at least one of the following: volume, aspect ratio, surface flatness, height, etc.

[0159] In some embodiments, a preset rule library can be established. After determining the geometric feature information corresponding to at least one fifth point cloud cluster, all point cloud clusters in the fifth point cloud cluster can be traversed based on this rule library, and the point cloud clusters that meet the preset rules in the fifth point cloud cluster can be screened out, so as to determine at least one target point cloud cluster.

[0160] In the above embodiments, by screening the fourth point cloud cluster representing the object that can be grasped through the second condition and removing the fourth point cloud cluster in at least one first point cloud cluster, the influence of the point cloud data of the object that can be grasped can be reduced when identifying the target obstacle, and the accuracy of the recognition result is improved.

[0161] In some embodiments, the target obstacle includes a dynamic obstacle; the target point cloud cluster includes a first target point cloud cluster corresponding to the dynamic obstacle; the above step S134 may include the following step S1345:

[0162] Step S1345, when the geometric feature information corresponding to the first point cloud cluster meets the third condition, determine the first point cloud cluster as the first target point cloud cluster; the third condition includes that the surface flatness is less than or equal to the third flatness threshold and the displacement speed within a continuous plurality of time frames is greater than or equal to the displacement speed threshold.

[0163] Here, the dynamic obstacle can be any suitable object, such as a person, a pet, etc. The first target point cloud cluster is the point cloud cluster corresponding to the dynamic obstacle. The number of first target point cloud clusters is at least one. In some embodiments, the number of first target point cloud clusters is less than or equal to the number of first point cloud clusters.

[0164] In some embodiments, the geometric feature information corresponding to the first target point cloud cluster satisfies a third condition. The third condition includes that the surface flatness is less than or equal to a third flatness threshold and the displacement speed within a plurality of consecutive time frames is greater than or equal to a displacement speed threshold. The third flatness threshold can be of any suitable magnitude, for example, 40%, 0.53, etc. The displacement speed threshold can be of any suitable magnitude, for example, 0.2 m / s (meters per second), 0.3 m / s, etc.

[0165] Exemplarily, the third condition includes that the surface flatness is less than or equal to 40% and the displacement speed within a plurality of consecutive time frames is greater than or equal to 0.2 m / s. The point cloud cluster in the first point cloud cluster whose geometric feature information satisfies the third condition can be used as the first target point cloud cluster. Among them, the object corresponding to the point cloud cluster in the first point cloud cluster that satisfies the surface flatness less than or equal to 40% and the displacement speed greater than or equal to 0.2 m / s within a plurality of consecutive time frames is a dynamic obstacle. It can be understood that a dynamic obstacle (such as a person or a pet, etc.) is in a moving state and usually does not have a large continuous plane and a low surface flatness. Therefore, through the joint limitation of the surface flatness and the displacement speed within a plurality of consecutive time frames, the first target point cloud cluster representing the dynamic obstacle in the first point cloud cluster can be accurately identified, improving the accuracy of the recognition result, and thus the dynamic obstacle can be accurately controlled in action to achieve obstacle avoidance processing.

[0166] In the above embodiment, when the geometric feature information corresponding to the first point cloud cluster satisfies the third condition, the first point cloud cluster is determined as the first target point cloud cluster corresponding to the dynamic obstacle, and the dynamic obstacle can be identified from the first point cloud cluster through the third condition.

[0167] In some embodiments, the target obstacle includes a static obstacle, and the target point cloud cluster includes a second target point cloud cluster corresponding to the static obstacle; the above step S134 may include the following step S1346:

[0168] Step S1346, when the geometric feature information corresponding to the first point cloud cluster satisfies a fourth condition, the first point cloud cluster is determined as the second target point cloud cluster; the fourth condition includes that the surface flatness is greater than or equal to a fourth flatness threshold and the volume is greater than or equal to a second volume threshold.

[0169] Here, the static obstacle can be any suitable object, for example, a stool, a tea table, a table, etc. The second target point cloud cluster is the point cloud cluster corresponding to the static obstacle. The number of the second target point cloud clusters is at least one. In some embodiments, the number of the second target point cloud clusters is less than or equal to the number of the first point cloud clusters.

[0170] In some embodiments, the geometric feature information corresponding to the second target point cloud cluster satisfies a fourth condition. The fourth condition includes that the surface flatness is greater than or equal to a fourth flatness threshold and the volume is greater than or equal to a second volume threshold. The fourth flatness threshold can be of any suitable size, for example, 75%, 0.6, etc. The second volume threshold is of any suitable size, for example, 0.5 m 3 , 0.6 m 3 , etc.

[0171] Exemplarily, the fourth condition includes that the surface flatness is greater than or equal to 75% and the volume is greater than or equal to 0.5 m 3 , and the point cloud clusters in the first point cloud cluster whose geometric feature information satisfies the fourth condition can be used as the second target point cloud clusters. Among them, the objects corresponding to the point cloud clusters in the first point cloud cluster that satisfy the surface flatness greater than or equal to 75% and the volume greater than or equal to 0.5 m 3 are static obstacles. It can be understood that static obstacles are in a stationary state and usually have a continuous plane (i.e., a relatively high surface flatness) and a large volume. Through the joint limitation of surface flatness and volume, the second target point cloud clusters representing static obstacles in the first point cloud cluster can be accurately identified, improving the accuracy of the recognition result, and then the static obstacles can be accurately controlled to perform obstacle avoidance processing.

[0172] In the above embodiment, when the geometric feature information corresponding to the first point cloud cluster satisfies the fourth condition, the first point cloud cluster is determined as the second target point cloud cluster corresponding to the static obstacle, and the static obstacle can be identified from the first point cloud cluster through the fourth condition.

[0173] In some embodiments, the above step S104 may include the following step S141:

[0174] Step S141, perform at least one of the following operations: control the fuselage and / or the robotic arm to stop moving; re-plan the movement path, and the movement path includes at least one of the following: the first movement path of the robotic arm, the second movement path of the fuselage; output an alarm message.

[0175] Here, the first movement path refers to the currently planned movement path of the robotic arm. The second movement path refers to the currently planned movement path of the fuselage.

[0176] The method for planning a movement path (including a first movement path and a second movement path) may include but is not limited to: re-planning the movement path by using a motion model, re-planning the movement path by multi-objective optimization, etc. For example, the movement path can be predicted by using a motion model. The motion model may include but is not limited to at least one of the following: Kalman filter model, particle filter model, etc. Again, for example, the problem of planning the movement path can be modeled as a multi-objective optimization problem, and then an optimization algorithm is used to solve the multi-objective optimization problem to re-plan the movement path. Among them, the objective may include avoiding a target obstacle. The optimization algorithm may include but is not limited to at least one of the following: genetic algorithm, particle swarm optimization algorithm, etc.

[0177] In some embodiments, when the movement path includes the first movement path of the robotic arm, the joint angles of the robotic arm can be adjusted by an inverse kinematics algorithm, so as to re-plan the first movement path. Among them, inverse kinematics can be understood as knowing the position and posture of the robotic arm and finding the joint angles of each joint of the robotic arm.

[0178] In some embodiments, when the safety state of the robotic arm indicates that there is a safety risk for the robotic arm, the fuselage and / or the robotic arm can be controlled to stop moving, then the movement path is re-planned, and then the movement is carried out according to the re-planned movement path.

[0179] In some embodiments, the fuselage includes a control component and a speaker. The control component has a control function and can send control instructions to the fuselage and / or the robotic arm to control the fuselage and / or the robotic arm to stop moving. At the same time, the control component can send an output instruction to the speaker to control the speaker to output an alarm message.

[0180] In some embodiments, when re-planning the movement path, the first movement path and the second movement path can be planned simultaneously, or according to the planning priority, the first movement path can be planned first, and then the second movement path can be planned, where the planning priority of the first movement path is higher than that of the second movement path.

[0181] In the above embodiments, various different operations can be performed in real time according to different actual scenarios, improving the intelligence of the action control method of the robot.

[0182] In some embodiments, the re-planning of the movement path is used to try to avoid the target obstacle. The above step S141 may include the following steps S1411 and step S1412:

[0183] Step S1411, when the re-planning is successful, control the robotic arm and / or the fuselage to act according to the re-planned movement path.

[0184] Here, the fuselage includes a control component and a chassis drive module, and the control component has control functions.

[0185] In some embodiments, in the case of successful replanning, the control component issues control instructions to the robotic arm or the fuselage, so that the robotic arm acts according to the replanned first movement path, and / or drives the fuselage to act according to the replanned second movement path through the chassis drive module.

[0186] Step S1412, in the case of failed replanning, control the fuselage and / or the robotic arm to stop moving, and / or output an alarm message.

[0187] Here, due to reasons such as space limitations, the replanning of the movement path may fail. Therefore, the fuselage and / or the robotic arm can be controlled to stop moving, and / or an alarm message can be output.

[0188] In some embodiments, the fuselage includes a control component and a speaker. The control component has control functions and can send control instructions to the fuselage and / or the robotic arm to control the fuselage and / or the robotic arm to stop moving. At the same time, the control component can send an output instruction to the speaker to control the speaker to output an alarm message.

[0189] In the above embodiments, different actions are performed according to whether the movement path planning is successful or failed, so that the target obstacle can be processed, and the obstacle avoidance requirement of the robotic arm for the target obstacle can be better met.

[0190] In some embodiments, in the case where the target obstacle is a dynamic obstacle, before performing step S141, step S140 can be performed:

[0191] Step S140, determine the safety state of the robotic arm. In the case where the safety state of the robotic arm indicates that there is a safety risk for the robotic arm, at least one of the following operations is performed.

[0192] Here, the safety state of the robotic arm can be used to indicate whether there is a safety risk for the robotic arm within the future target duration. The safety state of the robotic arm can indicate that there is a safety risk for the robotic arm, or can indicate that there is no safety risk for the robotic arm. The target duration can be of any suitable magnitude, for example, 3 s (seconds), 1.8 s, etc.

[0193] In some embodiments, in the case where the safety state of the robotic arm indicates that there is a safety risk for the robotic arm, the dynamic obstacle may intrude into the safety area of the robotic arm, and thus may collide or come into contact with the robotic arm. Therefore, at least one of the following operations can be performed: control the fuselage and / or the robotic arm to stop moving; replan the movement path, and the movement path includes at least one of the following: the first movement path of the robotic arm, the second movement path of the fuselage; output an alarm message.

[0194] In some embodiments, the safety state of the robotic arm can be determined based on the first trajectory information of the target obstacle within a target time period in the future and the second trajectory information of the robotic arm within the target time period in the future.

[0195] In some embodiments, according to the shapes of the target obstacle and the robotic arm, the target obstacle and the robotic arm can be respectively simplified into regular geometric bodies, and then the first trajectory information can be determined according to the change of the center of the regular geometric body corresponding to the target obstacle within the target time period in the future, and the second trajectory information can be determined according to the change of the center of the regular geometric body corresponding to the robotic arm within the target time period in the future.

[0196] In some embodiments, a first region can be determined based on the first trajectory information, and a second region can be determined based on the second trajectory information, and then the safety state of the robotic arm can be determined based on the first region and the second region. The first region refers to the region that the target obstacle may reach during the movement based on the first trajectory information. The second region refers to the region that the robotic arm may reach during the movement based on the second trajectory information.

[0197] In some embodiments, when determining the safety state of the robotic arm based on the first region and the second region, it can be determined whether there is an intersection region between the first region and the second region. When there is an intersection region between the first region and the second region, it is determined that the safety state of the robotic arm indicates that the robotic arm has a safety risk. When there is no intersection region between the first region and the second region, it is determined that the safety state of the robotic arm indicates that the robotic arm has no safety risk.

[0198] In some embodiments, when the safety state of the robotic arm indicates that the robotic arm has a safety risk, the fuselage and / or the robotic arm can be controlled to stop moving, then the movement path can be re-planned, and then the movement can be performed according to the re-planned movement path.

[0199] In some embodiments, when the safety state of the robotic arm indicates that the robotic arm has no safety risk, the dynamic obstacle will not invade the safety region of the robotic arm, and the safety state of the robotic arm can be continuously detected, so that when the safety state of the robotic arm indicates that the robotic arm has a safety risk, the fuselage and / or the robotic arm can be controlled to stop moving, and / or an alarm message can be output.

[0200] In the above embodiments, the safety state of the robotic arm is determined when the target obstacle includes a dynamic obstacle, so that various operations can be performed in real time according to different actual scenarios when the safety state of the robotic arm indicates that the robotic arm has a safety risk, improving the intelligence of the motion control method of the robot.

[0201] The following describes the application of the robot motion control method provided in the embodiments of the present application in an actual scenario, taking a floor-sweeping robot with a robotic arm as an example.

[0202] With the rapid development of smart home technology, robots have become an important tool for modern household cleaning. Traditional robots usually use infrared sensors, ultrasonic sensors, etc. to achieve obstacle avoidance and navigation. In complex environments (such as areas with dense furniture, dynamic obstacle scenarios), the obstacle avoidance of traditional robots has limitations. For example, infrared sensors are easily affected by ambient light interference, and ultrasonic sensors have large ranging errors in complex reflection scenarios. At the same time, traditional robots have a long data processing cycle and it is difficult to meet the real-time obstacle avoidance requirements of the robotic arm, resulting in problems such as low safety and poor flexibility.

[0203] Figure 2 Schematic of the implementation process of a robot motion control method provided in the embodiments of the present application Figure 2 As Figure 2 shown, the method may include the following steps S201 to step S216:

[0204] Step S201, start;

[0205] Step S202, start the TOF sensor (corresponding to the aforementioned acquisition component);

[0206] Step S203, the floor-sweeping robot performs data acquisition to generate first point cloud data;

[0207] Step S204, determine the second point cloud data based on the current pose information of the robotic arm and the load state of the robotic arm;

[0208] Step S205, preprocess the first point cloud data through filtering processing and coordinate calibration, and remove the second point cloud data from the preprocessed first point cloud data to obtain the third point cloud data;

[0209] Step S206, perform point cloud clustering analysis on the third point cloud data to obtain at least one point cloud cluster;

[0210] Step S207, determine whether there is a target point cloud cluster with the number of points greater than 100 in at least one point cloud cluster;

[0211] Here, if so, there is an obstacle, and enter step S208. If not, there is no obstacle, and enter step S216.

[0212] Step S208, continuously monitor for 100 ms;

[0213] Step S209, determine whether the surface flatness of the point cloud cluster is less than or equal to 70% and the displacement speed within a continuous plurality of time frames is greater than or equal to 0.2 m / s;

[0214] Here, if so, go to step S210; if not, go to step S212.

[0215] In step S210, the obstacle is a dynamic obstacle;

[0216] In step S211, determine whether the distance between the obstacle and the robotic arm is less than or equal to 10 cm;

[0217] Here, if so, go to step S215; if not, go to step S213.

[0218] In step S212, the obstacle is a static obstacle;

[0219] In step S213, determine whether the robotic arm and / or the fuselage can bypass;

[0220] Here, if so, go to step S214; if not, go to step S215.

[0221] In step S214, use inverse kinematics to re-plan the movement path of the robotic arm and / or the fuselage;

[0222] In step S215, control the fuselage and / or the robotic arm to stop moving, and at the same time, output an alarm message and wait for human intervention;

[0223] In step S216, end.

[0224] In the embodiments of the present application, the working area of the robotic arm is monitored in real time by the front TOF sensor. Since the TOF sensor can calculate the distance by measuring the time from the emission to the reception of the light pulse, it has the characteristics of high precision and high response speed, and can achieve precise object grasping and obstacle avoidance in a complex environment, improving the intelligent level and user experience of the sweeping robot.

[0225] Based on the above embodiments, the embodiments of the present application further provide an action control device for a robot. Figure 3 As shown in the structural schematic diagram of an action control device for a robot provided by the embodiments of the present application, Figure 3 As shown, the action control device 300 of the robot is applied to the robot. The robot includes a fuselage and a robotic arm. The robotic arm is installed in the accommodation bin on the top of the fuselage. The action control device 300 of the robot includes:

[0226] An acquisition module 301, configured to acquire first point cloud data of a target space; the target space includes the activity area of the robotic arm;

[0227] A processing module 302, configured to determine second point cloud data representing the robotic arm, and remove the second point cloud data from the first point cloud data to obtain third point cloud data;

[0228] The recognition module 303 is configured to determine whether there is an obstacle based on the third point cloud data and obtain a recognition result;

[0229] The control module 304 is configured to control the actions of the robotic arm and / or the fuselage based on the recognition result.

[0230] In some embodiments, the robot further includes an acquisition component disposed on the fuselage; the acquisition module 301 is further configured to control the acquisition component to acquire the first point cloud data of the target space.

[0231] In some embodiments, the recognition module 303 is further configured to determine that the third point cloud data meets the recognition condition; wherein, the recognition condition includes that the proportion of the point cloud data with a distance less than the distance threshold from the acquisition component in the third point cloud data is greater than or equal to a preset ratio; determine whether there is an obstacle based on the third point cloud data and obtain a recognition result.

[0232] In some embodiments, the processing module 302 is further configured to, when the load state of the robotic arm represents that the robotic arm grasps the target obstacle, determine the second point cloud data based on the fourth point cloud data corresponding to the current pose information of the robotic arm and the fifth point cloud data corresponding to the current pose information of the target obstacle; or, when the load state of the robotic arm represents that the robotic arm does not grasp the target obstacle, determine the second point cloud data based on the fourth point cloud data.

[0233] In some embodiments, the recognition module 303 is further configured to perform clustering processing on the third point cloud data to obtain at least one first point cloud cluster; determine at least one target point cloud cluster from the first point cloud clusters based on the geometric feature information corresponding to the at least one first point cloud cluster; and determine the target objects corresponding to the target point cloud clusters as target obstacles.

[0234] In some embodiments, the recognition module 303 is further configured to remove the second point cloud cluster representing the ground in the at least one first point cloud cluster to obtain at least one third point cloud cluster; the geometric feature information corresponding to the second point cloud cluster meets the first condition; the first condition includes that the surface flatness is greater than or equal to the first flatness threshold and the height is less than or equal to the height threshold; determine at least one target point cloud cluster from the third point cloud clusters based on the geometric feature information corresponding to the at least one third point cloud cluster.

[0235] In some embodiments, the recognition module 303 is further configured to remove a fourth point cloud cluster representing a target graspable object from at least one first point cloud cluster to obtain at least one fifth point cloud cluster; the geometric feature information corresponding to the fifth point cloud cluster satisfies a second condition; the second condition includes: the surface flatness is greater than or equal to a second flatness threshold, the volume is less than or equal to a first volume threshold, and the aspect ratio is less than or equal to a first aspect ratio threshold; based on the geometric feature information corresponding to at least one fifth point cloud cluster, at least one target point cloud cluster is determined from each fifth point cloud cluster.

[0236] In some embodiments, the target obstacle includes a dynamic obstacle; the target point cloud cluster includes a first target point cloud cluster corresponding to the dynamic obstacle; the recognition module 303 is further configured to determine the first point cloud cluster as the first target point cloud cluster when the geometric feature information corresponding to the first point cloud cluster satisfies a third condition; the third condition includes that the surface flatness is less than or equal to a third flatness threshold and the displacement speed within a continuous plurality of time frames is greater than or equal to a displacement speed threshold. In some embodiments,

[0237] In some embodiments, the target obstacle includes a static obstacle, and the target point cloud cluster includes a second target point cloud cluster corresponding to the static obstacle; the recognition module 303 is further configured to determine the first point cloud cluster as the second target point cloud cluster when the geometric feature information corresponding to the first point cloud cluster satisfies a fourth condition; the fourth condition includes that the surface flatness is greater than or equal to a fourth flatness threshold and the volume is greater than or equal to a second volume threshold.

[0238] In some embodiments, the control module 304 is further configured to perform at least one of the following operations: control the fuselage and / or the robotic arm to stop moving; re-plan a moving path, where the moving path includes at least one of the following: a first moving path of the robotic arm, a second moving path of the fuselage; output an alarm message.

[0239] In some embodiments, the control module 304 is further configured to re-plan the moving path to try to bypass the target obstacle, and further includes: when the re-planning is successful, control the robotic arm and / or the fuselage to act according to the re-planned moving path; when the re-planning fails, control the fuselage and / or the robotic arm to stop moving, and / or, output an alarm message.

[0240] In some embodiments, when the target obstacle is a dynamic obstacle, before performing the operation, the control module 304 is further configured to determine the safety state of the robotic arm, and when the safety state of the robotic arm indicates that there is a safety risk for the robotic arm, perform at least one of the operations.

[0241] Based on the above embodiments, an embodiment of the present application further provides a robot, Figure 4 Schematic diagram of the composition structure of a robot provided by an embodiment of the present applicationFigure 1 , as Figure 4 shown, the robot 400 includes: a fuselage 410, a robotic arm 420, a storage bin 430, and a control component 440, where:

[0242] The robotic arm 420 is installed in the storage bin 430 on top of the fuselage 420; and

[0243] The control component 440 is disposed inside the fuselage 410. The control component 440:

[0244] Obtains first point cloud data of a target space; the target space includes the active area of the robotic arm;

[0245] Determines second point cloud data representing the robotic arm, and removes the second point cloud data from the first point cloud data to obtain third point cloud data;

[0246] Judges whether there is an obstacle based on the third point cloud data to obtain an identification result;

[0247] Based on the identification result, controls the actions of the robotic arm and / or the fuselage.

[0248] In some embodiments, the robot further includes a collection component disposed on the fuselage; obtaining the first point cloud data of the target space includes: controlling the collection component to collect the first point cloud data of the target space.

[0249] Figure 5 The schematic diagram of the composition structure of a robot provided by an embodiment of the present application Figure 2 , the robot 400 includes: a fuselage 410, a robotic arm 420, a storage bin 430, a control component 440, and a collection component 450, where: the robotic arm 420 is installed in the storage bin 430 on top of the fuselage 420, the control component 440 is disposed inside the fuselage 410, and the collection component 450 is disposed in the front of the fuselage 410.

[0250] In some embodiments, the control component further determines that the third point cloud data meets the identification condition; where the identification condition includes that the proportion of the point cloud data with a distance less than the distance threshold between the third point cloud data and the collection component in the third point cloud data is greater than or equal to a preset ratio;

[0251] The control component also judges whether there is an obstacle based on the third point cloud data to obtain an identification result.

[0252] In some embodiments, when the load state of the robotic arm represents that the robotic arm grasps a target obstacle, the control component further determines the second point cloud data based on the fourth point cloud data corresponding to the current pose information of the robotic arm and the fifth point cloud data corresponding to the current pose information of the target obstacle;

[0253] When the load state of the robotic arm represents that the robotic arm does not grasp the target obstacle, the control component also determines the second point cloud data based on the fourth point cloud data.

[0254] In some embodiments, the recognition result includes at least one target obstacle; the control component also performs clustering processing on the third point cloud data to obtain at least one first point cloud cluster;

[0255] The control component also determines at least one target point cloud cluster from each of the first point cloud clusters based on the geometric feature information corresponding to the at least one first point cloud cluster;

[0256] The control component also determines the target object corresponding to each target point cloud cluster as the target obstacle.

[0257] In some embodiments, the control component also removes the second point cloud cluster representing the ground from the at least one first point cloud cluster to obtain at least one third point cloud cluster; the geometric feature information corresponding to the second point cloud cluster satisfies the first condition; the first condition includes that the surface flatness is greater than or equal to the first flatness threshold and the height is less than or equal to the height threshold;

[0258] The control component also determines at least one target point cloud cluster from each of the third point cloud clusters based on the geometric feature information corresponding to the at least one third point cloud cluster.

[0259] In some embodiments, the control component also removes the fourth point cloud cluster representing the target grabable object from the at least one first point cloud cluster to obtain at least one fifth point cloud cluster; the geometric feature information corresponding to the fifth point cloud cluster satisfies the second condition; the second condition includes: the surface flatness is greater than or equal to the second flatness threshold, the volume is less than or equal to the first volume threshold, and the aspect ratio is less than or equal to the first aspect ratio threshold;

[0260] The control component also determines at least one target point cloud cluster from each of the fifth point cloud clusters based on the geometric feature information corresponding to the at least one fifth point cloud cluster.

[0261] In some embodiments, the target obstacle includes a dynamic obstacle; the target point cloud cluster includes a first target point cloud cluster corresponding to the dynamic obstacle;

[0262] The control component also determines the first point cloud cluster as the first target point cloud cluster when the geometric feature information corresponding to the first point cloud cluster satisfies the third condition; the third condition includes that the surface flatness is less than or equal to the third flatness threshold and the displacement speed in a continuous plurality of time frames is greater than or equal to the displacement speed threshold.

[0263] In some embodiments, the target obstacle includes a static obstacle, and the target point cloud cluster includes a second target point cloud cluster corresponding to the static obstacle;

[0264] The control component also determines the first point cloud cluster as the second target point cloud cluster when the geometric feature information corresponding to the first point cloud cluster meets the fourth condition; the fourth condition includes that the surface flatness is greater than or equal to the fourth flatness threshold and the volume is greater than or equal to the second volume threshold.

[0265] In some embodiments, the control component also performs at least one of the following operations: controlling the fuselage and / or the robotic arm to stop moving;

[0266] re-planning the movement path, where the movement path includes at least one of the following: the first movement path of the robotic arm, the second movement path of the fuselage;

[0267] outputting an alarm message.

[0268] In some embodiments, when the re-planning is successful, the control component also controls the robotic arm and / or the fuselage to act according to the re-planned movement path;

[0269] When the re-planning fails, the control component also controls the fuselage and / or the robotic arm to stop moving, and / or outputs an alarm message.

[0270] In some embodiments, when the target obstacle is a dynamic obstacle, before performing the operation, the control component also determines the safety state of the robotic arm. If the safety state of the robotic arm indicates that there is a safety risk for the robotic arm, then at least one of the operations is performed.

[0271] Figure 6 Schematic diagram of the composition structure of a robot provided by an embodiment of the present application Figure 3 , as Figure 6 shown, the robot 400 includes: a fuselage 410, a robotic arm 420, a storage bin 430, a control component, and a collection component 450, where: the collection component 450 is arranged at the front of the fuselage 410, the robotic arm 420 is arranged on the top of the fuselage 410, the control component is arranged inside the fuselage 410, and the storage bin 430 is arranged on the top of the fuselage 410 for accommodating the robotic arm 420.

[0272] The description of the above embodiments of the robot and the robot's motion control device is similar to the description of the above method embodiments and has similar beneficial effects to the method embodiments. For the technical details not disclosed in the embodiments of the robot and the robot's motion control device of the present application, please refer to the description of the method embodiments of the present application for understanding.

[0273] It should be noted here that the descriptions of the above embodiments tend to emphasize the differences between the embodiments, and their similarities can be referred to each other. The descriptions of the above embodiments of the device, storage medium, computer program, and computer program product are similar to the descriptions of the above method embodiments and have beneficial effects similar to those of the method embodiments. For the technical details not disclosed in the embodiments of the device, storage medium, computer program, and computer program product of the present application, please refer to the descriptions of the method embodiments of the present application for understanding.

[0274] It should be noted that in the embodiments of the present application, if the above method is implemented in the form of software function modules and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of the present application, in essence, or the part that contributes to the related technology, can be embodied in the form of a software product. The software product is stored in a storage medium and includes several instructions for causing an electronic device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in the embodiments of the present application. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, ROMs, magnetic disks, or optical discs that can store program codes. In this way, the embodiments of the present application are not limited to any specific combination of hardware and software.

[0275] The embodiments of the present application provide a computer device, including a memory and a processor. The memory stores a computer program that can run on the processor, and when the processor executes the program, it implements some or all of the steps in the above method.

[0276] The embodiments of the present application provide a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the above method. The computer-readable storage medium can be transient or non-transient.

[0277] The embodiments of the present application provide a computer program product. The computer program product includes a non-transient computer-readable storage medium storing a computer program. When the computer program is read and executed by a computer, it implements some or all of the steps in the above method. The computer program product can be specifically implemented in a manner of hardware, software, or a combination thereof. In an optional embodiment, the computer program product is specifically embodied as a computer storage medium. In another optional embodiment, the computer program product is specifically embodied as a software product, such as a Software Development Kit (SDK), etc.

[0278] An embodiment of the present application provides a computer program product, including a computer program or instruction, which, when executed by a processor, implements some or all of the steps in the above-mentioned motion control method of the robot.

[0279] An embodiment of the present application provides a processor, which is communicatively connected to a memory. The memory stores a computer program that can run on the processor. When the processor executes the computer program, it implements the steps in the above-mentioned motion control method of the robot.

[0280] An embodiment of the present application provides a robot, including a processor and a memory. The memory stores a computer program that can run on the processor. When the processor executes the computer program, it implements the steps in the above-mentioned motion control method of the robot.

[0281] It should be noted here that the descriptions of the above embodiments of the robot, storage medium, device, and program product are similar to those of the above method embodiments and have similar beneficial effects to the method embodiments. For the technical details not disclosed in the embodiments of the robot, storage medium, device, and program product of the present application, please refer to the descriptions of the method embodiments of the present application for understanding.

[0282] It should be understood that the "one embodiment" or "an embodiment" mentioned throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of the present application. Therefore, the appearances of "in one embodiment" or "in an embodiment" throughout the specification do not necessarily refer to the same embodiment. In addition, these specific features, structures, or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present application, the magnitudes of the serial numbers of the above steps / processes do not mean the order of execution. The execution order of each step / process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application. The serial numbers of the embodiments of the present application above are only for description and do not represent the advantages or disadvantages of the embodiments.

[0283] It should be noted that in this article, the term "comprising", "including" or any other variation thereof is intended to cover a non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the phrase "including a..." does not exclude the existence of additional identical elements in the process, method, article or device including the element.

[0284] In several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed with each other can be through some interfaces. The indirect coupling or communication connection of the devices or units can be electrical, mechanical, or other forms.

[0285] The units described above as separate components may or may not be physically separated. The components shown as units may or may not be physical units; they can be located in one place or distributed to multiple network units; some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0286] In addition, each functional unit in the embodiments of this application can be all integrated in a processing unit, or each unit can be separately used as a unit, or two or more units can be integrated in one unit; the above integrated unit can be implemented in the form of hardware, or in the form of hardware plus software functional units.

[0287] Those of ordinary skill in the art can understand that all or part of the steps to implement the above method embodiments can be completed by hardware related to program instructions. The foregoing program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps including the above method embodiments; and the foregoing storage medium includes: removable storage devices, read-only memories, magnetic disks, or optical disks and other various media that can store program codes.

[0288] Alternatively, if the above integrated unit of this application is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of this application essentially or the part that contributes to the related technology can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in the embodiments of this application. And the foregoing storage medium includes: removable storage devices, ROMs, magnetic disks, or optical disks and other various media that can store program codes.

[0289] As described above, it is only the implementation mode of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application can easily think of changes or substitutions, which should all be covered within the protection scope of the present application.

Claims

1. A robot motion control method, characterized in that: The robot comprises a body and a mechanical arm, wherein the mechanical arm is arranged on the body, and the method comprises: Acquire first point cloud data of a target space; the target space includes an activity area of ​​the robotic arm; Determine second point cloud data representing the robotic arm, and remove the second point cloud data from the first point cloud data to obtain third point cloud data; Determine whether there is an obstacle based on the third point cloud data, and obtain a recognition result; Based on the recognition result, the action of the robot arm and / or the body is controlled.

2. The robot motion control method according to claim 1, characterized in that: The robot also includes a collection component disposed on the body; Get the first point cloud data of the target space, including: The acquisition component is controlled to acquire first point cloud data of the target space.

3. The robot motion control method according to claim 2, characterized in that: Determining whether there is an obstacle based on the third point cloud data to obtain a recognition result includes: Determining that the third point cloud data satisfies an identification condition; wherein the identification condition includes that the proportion of point cloud data in the third point cloud data whose distance from the acquisition component is less than a distance threshold in the third point cloud data is greater than or equal to a preset ratio; Whether there is an obstacle is determined based on the third point cloud data to obtain the recognition result.

4. The robot motion control method according to claim 1, characterized in that: Determining second point cloud data characterizing the robotic arm based on the current posture information of the robotic arm and the load state of the robotic arm includes: In a case where the load state of the robotic arm represents that the robotic arm grasps a target obstacle, the second point cloud data is determined based on fourth point cloud data corresponding to current posture information of the robotic arm and fifth point cloud data corresponding to current posture information of the target obstacle; or In a case where the load state of the robotic arm indicates that the robotic arm has not grasped the target obstacle, the second point cloud data is determined based on the fourth point cloud data.

5. The robot motion control method according to any one of claims 1 to 4, characterized in that: Determining whether there is an obstacle based on the third point cloud data to obtain a recognition result includes: Performing clustering processing on the third point cloud data to obtain at least one first point cloud cluster; Based on the geometric feature information corresponding to the at least one first point cloud cluster, determining at least one target point cloud cluster from each of the first point cloud clusters; The target object corresponding to each of the target point cloud clusters is determined as the target obstacle.

6. The robot motion control method according to claim 5, characterized in that: Based on the geometric feature information corresponding to the at least one first point cloud cluster, determining at least one target point cloud cluster from each of the first point cloud clusters includes: Eliminate a second point cloud cluster representing the ground from the at least one first point cloud cluster to obtain at least one third point cloud cluster; the geometric feature information corresponding to the second point cloud cluster satisfies a first condition; the first condition includes that the surface flatness is greater than or equal to a first flatness threshold and the height is less than or equal to a height threshold; At least one target point cloud cluster is determined from each of the third point cloud clusters based on the geometric feature information corresponding to the at least one third point cloud cluster.

7. The robot motion control method according to claim 5, characterized in that: Based on the geometric feature information corresponding to the at least one first point cloud cluster, determining at least one target point cloud cluster from each of the first point cloud clusters includes: Eliminate the fourth point cloud cluster representing the target graspable object from the at least one first point cloud cluster to obtain at least one fifth point cloud cluster; the geometric feature information corresponding to the fifth point cloud cluster satisfies the second condition; the second condition includes: the surface flatness is greater than or equal to the second flatness threshold, the volume is less than or equal to the first volume threshold, and the aspect ratio is less than or equal to the first aspect ratio threshold; At least one target point cloud cluster is determined from each of the fifth point cloud clusters based on the geometric feature information corresponding to the at least one fifth point cloud cluster.

8. The robot motion control method according to claim 5, characterized in that: The target obstacle includes a dynamic obstacle; The target point cloud cluster includes a first target point cloud cluster corresponding to the dynamic obstacle; Based on the geometric feature information corresponding to the at least one first point cloud cluster, determining at least one target point cloud cluster from each of the first point cloud clusters includes: When the geometric feature information corresponding to the first point cloud cluster satisfies a third condition, determining the first point cloud cluster as the first target point cloud cluster; The third condition includes that the surface flatness is less than or equal to a third flatness threshold and the displacement speed in a plurality of consecutive time frames is greater than or equal to a displacement speed threshold.

9. The robot motion control method according to claim 5, characterized in that: The target obstacle includes a static obstacle, and the target point cloud cluster includes a second target point cloud cluster corresponding to the static obstacle; Based on the geometric feature information corresponding to the at least one first point cloud cluster, determining at least one target point cloud cluster from each of the first point cloud clusters includes: When the geometric feature information corresponding to the first point cloud cluster satisfies a fourth condition, determining the first point cloud cluster as the second target point cloud cluster; The fourth condition includes that the surface flatness is greater than or equal to a fourth flatness threshold and the volume is greater than or equal to a second volume threshold.

10. The robot motion control method according to claim 8 or 9, characterized in that: Based on the recognition result, controlling the action of the robot arm and / or the body includes: Do at least one of the following: Controlling the fuselage and / or the robotic arm to stop moving; Replanning a moving path, wherein the moving path includes at least one of the following: a first moving path of the robot arm and a second moving path of the body; Output alarm information.

11. The robot motion control method according to claim 10, characterized in that: The replanning of the movement path to try to bypass the target obstacle further includes: If the re-planning is successful, controlling the robot arm and / or the body to move according to the re-planned movement path; In case of re-planning failure, the fuselage and / or the robotic arm are controlled to stop moving, and / or an alarm message is output.

12. The robot motion control method according to claim 10, characterized in that: In the case where the target obstacle is a dynamic obstacle, before performing the operation, the method further includes: The safety status of the robotic arm is determined, and when the safety status of the robotic arm indicates that the robotic arm has a safety risk, at least one of the operations is performed.

13. A robot motion control device, characterized in that: The robot comprises a body and a mechanical arm, wherein the mechanical arm is arranged on the body, and the device comprises: An acquisition module, used for acquiring first point cloud data of a target space; the target space includes an activity area of ​​the robotic arm; A processing module, determining second point cloud data representing the robotic arm, and removing the second point cloud data from the first point cloud data to obtain third point cloud data; An identification module, used to determine whether there is an obstacle based on the third point cloud data, and obtain an identification result; A control module is used to control the movement of the robotic arm and / or the body based on the recognition result.

14. The robot motion control device according to claim 13, characterized in that: The robot also includes a collection component disposed on the body; The acquisition module is further used to control the acquisition component to acquire the first point cloud data of the target space.

15. The robot motion control device according to claim 14, characterized in that: The recognition module is further used to determine whether the third point cloud data meets the recognition conditions; wherein the recognition conditions include that the proportion of point cloud data in the third point cloud data whose distance to the acquisition component is less than a distance threshold in the third point cloud data is greater than or equal to a preset ratio; based on the third point cloud data, it is determined whether there is an obstacle to obtain the recognition result.

16. The robot motion control device according to claim 13, characterized in that: The processing module is further used to determine the second point cloud data based on fourth point cloud data corresponding to current posture information of the robotic arm and fifth point cloud data corresponding to current posture information of the target obstacle when the load state of the robotic arm indicates that the robotic arm grasps the target obstacle; or, to determine the second point cloud data based on the fourth point cloud data when the load state of the robotic arm indicates that the robotic arm does not grasp the target obstacle.

17. The robot motion control device according to any one of claims 13 to 16, characterized in that: The recognition module is further configured to perform clustering processing on the third point cloud data to obtain at least one first point cloud cluster; and determine at least one target point cloud cluster from each of the first point cloud clusters based on geometric feature information corresponding to the at least one first point cloud cluster; The target object corresponding to each of the target point cloud clusters is determined as the target obstacle.

18. The robot motion control device according to claim 17, characterized in that: The recognition module is also used to eliminate the second point cloud cluster representing the ground from the at least one first point cloud cluster to obtain at least one third point cloud cluster; the geometric feature information corresponding to the second point cloud cluster satisfies a first condition; the first condition includes that the surface flatness is greater than or equal to a first flatness threshold and the height is less than or equal to a height threshold; based on the geometric feature information corresponding to the at least one third point cloud cluster, determine at least one target point cloud cluster from each of the third point cloud clusters.

19. The robot motion control device according to claim 17, characterized in that: The recognition module is further used to eliminate the fourth point cloud cluster representing the target graspable object in the at least one first point cloud cluster to obtain at least one fifth point cloud cluster; the geometric feature information corresponding to the fifth point cloud cluster satisfies the second condition; the second condition includes: the surface flatness is greater than or equal to the second flatness threshold, the volume is less than or equal to the first volume threshold, and the aspect ratio is less than or equal to the first aspect ratio threshold; based on the geometric feature information corresponding to the at least one fifth point cloud cluster, at least one target point cloud cluster is determined from each of the fifth point cloud clusters.

20. The robot motion control device according to claim 17, characterized in that: The target obstacle includes a dynamic obstacle; the target point cloud cluster includes a first target point cloud cluster corresponding to the dynamic obstacle; The recognition module is further used to determine the first point cloud cluster as the first target point cloud cluster when the geometric feature information corresponding to the first point cloud cluster satisfies a third condition; the third condition includes that the surface flatness is less than or equal to a third flatness threshold and the displacement speed in multiple consecutive time frames is greater than or equal to a displacement speed threshold.

21. The robot motion control device according to claim 17, characterized in that: The target obstacle includes a static obstacle, and the target point cloud cluster includes a second target point cloud cluster corresponding to the static obstacle; The recognition module is further used to determine the first point cloud cluster as the second target point cloud cluster when the geometric feature information corresponding to the first point cloud cluster satisfies a fourth condition; the fourth condition includes that the surface flatness is greater than or equal to a fourth flatness threshold and the volume is greater than or equal to a second volume threshold.

22. The robot motion control device according to claim 20 or 21, characterized in that: The control module is used to perform at least one of the following operations: control the fuselage and / or the robotic arm to stop moving; re-plan a moving path, the moving path including at least one of the following: a first moving path of the robotic arm, a second moving path of the fuselage; and output an alarm message.

23. The robot motion control device according to claim 21, characterized in that: The control module is used to re-plan the moving path to try to bypass the target obstacle, and also includes: if the re-planning is successful, controlling the robotic arm and / or the fuselage to move according to the re-planned moving path; if the re-planning fails, controlling the fuselage and / or the robotic arm to stop moving, and / or outputting an alarm message.

24. The robot motion control device according to claim 21, characterized in that: In the case where the target obstacle is a dynamic obstacle, before performing the operation, The control module is further configured to determine a safety status of the robotic arm, and to execute at least one of the operations when the safety status of the robotic arm indicates that there is a safety risk to the robotic arm.

25. A robot, characterized in that: The method comprises a processor and a memory, wherein the memory stores a computer program executable on the processor, and the processor implements the method according to any one of claims 1 to 12 when executing the computer program.

26. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor, the steps in the method according to any one of claims 1 to 12 are implemented.

27. A computer program product, characterized in that The method comprises a computer program or an instruction, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 12.

Images