Method and device for recognizing three-dimensional spatial pose of objects for grasping by a robotic arm

Abstract

The present application relates to a method and device for recognizing the three-dimensional spatial posture of an object grasped by a robotic arm, which recognizes and obtains the front frame point cloud and the back frame point cloud of the object to be grasped; downsamples the front frame point cloud and the back frame point cloud data respectively to obtain the front and back frame sampling point clouds; searches for the front and back frame sampling point clouds, and obtains the relative transformation point clouds from the front and back frame sampling point clouds according to the preset extraction conditions; searches for the transformation point cloud and the back frame point cloud, and obtains the final transformation point cloud from the transformation point cloud and the back frame point cloud according to the preset radius; recognizes and processes the final transformation point cloud, adds the recognized three-dimensional spatial posture result to the cache list, selects the best three-dimensional spatial posture result from the cache list and sends it to the robotic arm. It effectively utilizes the front and back frame changes of the point cloud data, takes into account the recognition accuracy and speed, and improves the efficiency of the robotic arm grasping recognition. It solves the problem of unnecessary computing burden caused by redundant recognition and posture screening.

Description

Technical Field

[0001] The present disclosure relates to the field of three-dimensional vision technology, and in particular to a method, device, control system and non-volatile computer-readable storage medium for recognizing the three-dimensional spatial posture of an object grasped by a robotic arm. Background Art

[0002] As a replacement for manual repetitive operations, robotic arms are widely used in industrial scenarios, among which the most common task is grasping and assembly. Good grasping performance is the prerequisite for precise assembly, and in addition to recognition accuracy, recognition speed is also an important indicator in industrial production.

[0003] In the existing technology, the technical means of obtaining point clouds through three-dimensional camera recognition has high adaptability to industrial lighting conditions and rich geometric expression of object spatial information, so that it can be combined with robots to further improve the accuracy of the recognition process during grasping.

[0004] However, in the robot grasping scene, the target objects are often random, scattered, and stacked, and the robot grasps only one or a few objects at a time. Most of the target objects do not change their spatial position and remain in place. In addition, the point cloud of the grasping scene output by the 3D camera has a large amount of data. Most existing methods directly use all the data of each frame of the point cloud for recognition, which not only brings a heavy burden to the computer processing, but also greatly slows down the time of feature extraction, directly affecting the efficiency of the robot grasping. Summary of the invention

[0005] In view of this, the purpose of the present invention is to solve the problem of unnecessary computational burden caused by redundant recognition and posture screening in the existing grasping recognition method technology, and compared with the traditional direct recognition of all target objects in the scene point cloud, it has the advantages of high detection accuracy and high detection efficiency. The present invention proposes a three-dimensional spatial posture recognition method, device, control system and non-volatile computer-readable storage medium for objects grasped by a robot arm, which effectively utilizes the changes in the previous and next frames of point cloud data, while taking into account the recognition accuracy and speed, and improves the recognition efficiency of the robot arm grasping.

[0006] According to one aspect of the present disclosure, a method for recognizing the three-dimensional spatial position of an object grasped by a robotic arm is provided, comprising the following steps:

[0007] S100, identifying and acquiring the previous frame point cloud and the next frame point cloud of the object to be grasped;

[0008] S200, down-sampling the previous frame point cloud and the next frame point cloud data respectively to obtain the previous and next frame sampling point clouds;

[0009] S300, searching for the sampling point clouds of the previous and next frames, and obtaining relative transformation point clouds from the sampling point clouds of the previous and next frames according to a preset extraction condition;

[0010] S400, searching the transformed point cloud and the subsequent frame point cloud, and obtaining a final transformed point cloud from the transformed point cloud and the subsequent frame point cloud according to a preset radius;

[0011] S500, performing recognition processing on the final transformed point cloud, adding the recognized three-dimensional space pose result to a cache list, selecting the best three-dimensional space pose result in the cache list and sending it to the robotic arm.

[0012] As a preferred implementation scheme of the present application, optionally, in step S100, after identifying and acquiring the previous frame point cloud and the next frame point cloud of the object to be grasped, it also includes:

[0013] S101, preset filtering conditions;

[0014] S102, removing invalid point data in the previous frame point cloud and the next frame point cloud according to a preset filtering condition; and

[0015] S103 , filtering to remove outlier noise in the previous frame point cloud and the next frame point cloud.

[0016] As a preferred implementation scheme of the present application, optionally, in step S300, searching the sampling point clouds of the previous and next frames, and obtaining the relative transformation point clouds from the sampling point clouds of the previous and next frames according to a preset extraction condition, includes:

[0017] S310, obtaining and searching the sampling point clouds of the previous and next frames;

[0018] S320, extracting the nearest point of the sampling point cloud of the subsequent frame in the sampling point cloud of the previous frame according to the preset extraction condition;

[0019] S330 , collecting the nearest point to obtain a relative transformation point cloud of the subsequent frame sampling point cloud relative to the previous frame sampling point cloud.

[0020] As a preferred implementation scheme of the present application, optionally, in step S400, searching the transformed point cloud and the subsequent frame point cloud, and obtaining the final transformed point cloud from the transformed point cloud and the subsequent frame point cloud according to a preset radius, includes:

[0021] S410, acquiring the relative transformation point cloud, and performing noise reduction processing on the relative transformation point cloud to obtain a noise reduction point cloud;

[0022] S420, acquiring and searching the noise reduction point cloud and the subsequent frame point cloud, and extracting points within the preset radius from the noise reduction point cloud and the subsequent frame point cloud respectively;

[0023] S430: Collect points within the preset radius, and collect and obtain the final transformed point cloud.

[0024] According to another aspect of the present disclosure, there is also provided a device for implementing the above-mentioned method for recognizing the three-dimensional spatial posture of an object grasped by a robotic arm, comprising a point cloud acquisition module, a downsampling processing module, a relative transformation point cloud acquisition module, a final transformation point cloud acquisition module and a recognition processing module, wherein:

[0025] The point cloud acquisition module is used to identify and acquire the previous frame point cloud and the next frame point cloud of the object to be grasped;

[0026] The downsampling processing module is used to perform downsampling processing on the previous frame point cloud and the next frame point cloud data respectively to obtain the previous and next frame sampling point clouds;

[0027] The relative transformation point cloud acquisition module is used to search the sampling point clouds of the previous and next frames, and obtain the relative transformation point clouds from the sampling point clouds of the previous and next frames according to preset extraction conditions;

[0028] The final transformed point cloud acquisition module is used to search the transformed point cloud and the subsequent frame point cloud, and obtain the final transformed point cloud from the transformed point cloud and the subsequent frame point cloud according to a preset radius;

[0029] The recognition processing module is used to perform recognition processing on the final transformed point cloud, add the recognized three-dimensional space pose result to a cache list, select the best three-dimensional space pose result in the cache list and send it to the robotic arm.

[0030] As a preferred implementation scheme of the present application, optionally, it further includes a first noise reduction module,

[0031] The first noise reduction module is used to preset filtering conditions; and, according to the preset filtering conditions, remove invalid point data in the previous frame point cloud and the next frame point cloud; and, filter to remove outlier noise in the previous frame point cloud and the next frame point cloud.

[0032] As a preferred implementation scheme of the present application, optionally, the relative transformation point cloud acquisition module includes:

[0033] Sampling acquisition module: used to acquire and search the sampling point clouds of the previous and next frames;

[0034] The first extraction module is used to extract the nearest point of the sampling point cloud of the subsequent frame in the sampling point cloud of the previous frame according to the preset extraction conditions;

[0035] The first point collection module is used to collect the nearest point and obtain the relative transformation point cloud of the subsequent frame sampling point cloud relative to the previous frame sampling point cloud.

[0036] As a preferred implementation scheme of the present application, optionally, the final transformed point cloud acquisition module includes:

[0037] A second noise reduction module is used to obtain the relative transformation point cloud, and perform noise reduction processing on the relative transformation point cloud to obtain a noise reduction point cloud;

[0038] A second extraction module: used to obtain and search the noise reduction point cloud and the subsequent frame point cloud, and extract points within the preset radius from the noise reduction point cloud and the subsequent frame point cloud respectively;

[0039] The second point collection module is used to collect points within the preset radius, collect and obtain the final transformed point cloud.

[0040] According to another aspect of the present disclosure, there is also provided a control system, comprising:

[0041] processor;

[0042] a memory for storing processor-executable instructions;

[0043] Wherein, the processor is configured to implement the above-mentioned three-dimensional spatial posture recognition method for object grasping by a robotic arm when executing the executable instructions.

[0044] According to another aspect of the present disclosure, a non-volatile computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the above-mentioned method for recognizing the three-dimensional spatial posture of an object grasped by a robotic arm is implemented.

[0045] Technical effects of this application:

[0046] The present invention identifies and obtains the front frame point cloud and the back frame point cloud of the object to be grasped; performs downsampling processing on the front frame point cloud and the back frame point cloud data respectively to obtain the front and back frame sampling point clouds; searches for the front and back frame sampling point clouds, and obtains the relative transformation point cloud from the front and back frame sampling point clouds according to the preset extraction conditions; searches for the transformation point cloud and the back frame point cloud, and obtains the final transformation point cloud from the transformation point cloud and the back frame point cloud according to the preset radius; performs recognition processing on the final transformation point cloud, adds the recognized three-dimensional space posture result to the cache list, selects the best three-dimensional space posture result from the cache list and sends it to the robot arm. The front and back frame changes of the point cloud data are effectively utilized, while taking into account the recognition accuracy and speed, and improving the grasping recognition efficiency of the robot arm. At the same time, the problem of unnecessary computing burden caused by redundant recognition and posture screening is solved. Compared with direct recognition on all data of a frame of point cloud, the redundant and time-consuming calculation caused by repeated recognition of multiple frames of relatively static targets has a more efficient recognition speed. At the same time, a caching mechanism for the three-dimensional spatial pose results is introduced to continuously update and select the best grasping pose. This will significantly improve efficiency when the scene contains more target objects.

[0047] Further features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the disclosure and, together with the description, serve to explain the principles of the disclosure.

[0049] Figure 1 The figure shows a schematic diagram of the implementation process of the method for recognizing the three-dimensional spatial posture of an object grasped by a robot arm according to the present invention;

[0050] Figure 2 Shown is a schematic diagram of the operation of the robot arm of the present invention performing a grasping action. DETAILED DESCRIPTION

[0051] Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0052] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0053] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present disclosure.

[0054] Example 1

[0055] like Figure 1 The figure is a schematic diagram of the implementation process of the present invention.

[0056] According to one aspect of the present disclosure, a method for recognizing the three-dimensional spatial position of an object grasped by a robotic arm is provided, comprising the following steps:

[0057] S100, identifying and acquiring the previous frame point cloud and the next frame point cloud of the object to be grasped;

[0058] The present invention effectively utilizes the changes in the previous and next frames of point cloud data, while taking into account both recognition accuracy and speed, continuously acquiring the point cloud data of the objects to be recognized in the previous and next frames, and repeatedly recognizing relatively static targets in multiple frames. Compared with direct recognition on all the data of a frame of point cloud, the redundant and time-consuming calculations brought about have a more efficient recognition speed. At the same time, a caching mechanism for the three-dimensional spatial posture results is introduced to continuously update and select the best grasping posture, which is more significantly improved in efficiency when the scene contains more target objects.

[0059] In this embodiment, the point cloud of the previous frame and the point cloud of the next frame of the object to be grasped, that is, the point cloud data of the previous frame and the point cloud of the next frame, can be obtained by a three-dimensional camera. In this embodiment, the point cloud of the previous frame and the point cloud of the next frame obtained are point cloud data of two consecutive frames, such as the point cloud of the first second and the point cloud of the second second.

[0060] S200, down-sampling the previous frame point cloud and the next frame point cloud data respectively to obtain the previous and next frame sampling point clouds;

[0061] After obtaining the point cloud data, since it is coarse data, it is necessary to downsample the previous frame point cloud and the next frame point cloud data respectively to obtain the previous and next frame sampled point clouds; on the premise of ensuring the recognition accuracy, appropriate downsampling is performed to obtain the sampled point cloud.

[0062] S300, searching for the sampling point clouds of the previous and next frames, and obtaining relative transformation point clouds from the sampling point clouds of the previous and next frames according to a preset extraction condition;

[0063] The relative transformation point cloud is the relative change point cloud data of the later frame point cloud relative to the previous frame point cloud. The KD tree structure and k-NN search can be used to extract the closest point of the later frame point cloud in the previous frame point cloud, that is, to extract the point cloud data of the later frame point cloud closest to the previous frame point cloud, and roughly obtain the part of the later frame point cloud data that has changed compared to the previous frame point cloud data, which is used as the relative transformation point cloud.

[0064] S400, searching the transformed point cloud and the subsequent frame point cloud, and obtaining a final transformed point cloud from the transformed point cloud and the subsequent frame point cloud according to a preset radius;

[0065] After the relative transformation point cloud is obtained, the relative transformation point cloud and the post-frame point cloud are processed twice, and the point cloud within a certain radius is extracted as the final point cloud data. That is, the KD tree structure is used to extract the points within a certain radius of each point in the post-frame point cloud and the relative transformation point cloud, and the points of the post-frame point cloud and the relative transformation point cloud within this radius are obtained to capture the changed parts of the two frames as comprehensively as possible as the final transformation point cloud.

[0066] S500, performing recognition processing on the final transformed point cloud, adding the recognized three-dimensional space pose result to a cache list, selecting the best three-dimensional space pose result in the cache list and sending it to the robotic arm.

[0067] The object recognition algorithm is processed on the final transformed point cloud to realize the recognition of all target objects in the transformed part of the point cloud; the recognized three-dimensional spatial pose results are added to the cache list and l The best 3D spatial pose result is selected and sent to the robot controller to perform operations such as grabbing. The remaining 3D spatial pose results remain in the cache list. l middle.

[0068] Therefore, the present invention can improve the efficiency of robot arm grasping and recognition by taking into account the recognition accuracy and speed by utilizing the changes of the previous and next frames of point cloud data. Compared with direct recognition on all the data of a frame of point cloud, the redundant and time-consuming calculations brought about by repeated recognition of relatively static targets in multiple frames have a more efficient recognition speed. At the same time, a caching mechanism for the three-dimensional space posture results is introduced to continuously update and select the best grasping posture, which is more obvious in the case where the scene contains more target objects.

[0069] like Figure 2 As shown, it is a schematic diagram of the specific point cloud operation when the robot arm of the present invention performs grasping.

[0070] As an implementation of this embodiment, select the first frame point cloud P 1 ， And the second frame point cloud P2, as the front frame point cloud and back frame point cloud data of this embodiment.

[0071] As a preferred implementation scheme of the present application, optionally, in step S100, after identifying and acquiring the previous frame point cloud and the next frame point cloud of the object to be grasped, it also includes:

[0072] S101, preset filtering conditions;

[0073] The filtering conditions are set according to user-defined settings to remove noise.

[0074] S102, removing invalid point data in the previous frame point cloud and the next frame point cloud according to a preset filtering condition; and

[0075] S103 , filtering to remove outlier noise in the previous frame point cloud and the next frame point cloud.

[0076] First, get the first frame of point cloud from the 3D camera P 1; Yes P 1. Remove the data corresponding to the invalid points and remove the outlier noise caused by the 3D camera acquisition through filtering.

[0077] For the first frame of point cloud, continue processing as follows:

[0078] Under the premise of ensuring recognition accuracy, appropriate downsampling processing is performed on P1 to obtain the sampling point cloud P 1 d ;exist P 1 d The object recognition algorithm is processed on the scene to realize the recognition of all target objects in the scene and obtain the three-dimensional spatial pose results.

[0079] The best 3D spatial pose result among all the identified targets is selected and sent to the robot controller to perform operations such as grabbing, while the results of the remaining targets are placed in the cache list l. The selection of the best 3D spatial pose result depends on many situations, including:

[0080] (1) Select all target objects and prioritize them from top to bottom according to their positions in the loading area to avoid the position change of other targets caused by the departure of the grasped object;

[0081] (2) Under a condition, sequentially traverse and select all target objects that are closest to the end gripper, so that the robot arm can reach the target object as quickly as possible;

[0082] (3) Determine the corresponding position and posture of the target object selected in step 2 to determine whether the robot body and the end gripper will collide with other objects in the environment. If there is no collision, perform the grasping operation. Otherwise, traverse step 2 again.

[0083] After processing the first frame of point cloud, the capture is performed.

[0084] As a preferred implementation scheme of the present application, optionally, in step S300, searching the sampling point clouds of the previous and next frames, and obtaining the relative transformation point clouds from the sampling point clouds of the previous and next frames according to a preset extraction condition, includes:

[0085] S310, obtaining and searching the sampling point clouds of the previous and next frames;

[0086] S320, extracting the nearest point of the sampling point cloud of the subsequent frame in the sampling point cloud of the previous frame according to the preset extraction condition;

[0087] S330 , collecting the nearest point to obtain a relative transformation point cloud of the subsequent frame sampling point cloud relative to the previous frame sampling point cloud.

[0088] For the second frame point cloud:

[0089] Get the second frame point cloud from the 3D camera P 2. For P 2. First, remove the data corresponding to the invalid points, and then remove the outlier noise caused by the 3D camera acquisition through filtering;

[0090] Under the premise of ensuring recognition accuracy, P 2 Take appropriate downsampling to get P 2 d ; Use KD tree structure and k-NN search to extract P 2 d exist P 1 d The nearest point among all the points roughly obtains the part of the point cloud data of the second frame compared with the first frame as the relative transformation point cloud P 21 d ;

[0091] As a preferred implementation scheme of the present application, optionally, in step S400, searching the transformed point cloud and the subsequent frame point cloud, and obtaining the final transformed point cloud from the transformed point cloud and the subsequent frame point cloud according to a preset radius, includes:

[0092] S410, acquiring the relative transformation point cloud, and performing noise reduction processing on the relative transformation point cloud to obtain a noise reduction point cloud;

[0093] S420, acquiring and searching the noise reduction point cloud and the subsequent frame point cloud, and extracting points within the preset radius from the noise reduction point cloud and the subsequent frame point cloud respectively;

[0094] S430: Collect points within the preset radius, and collect and obtain the final transformed point cloud.

[0095] Relative transformation point cloudP 21 d There are point clouds with small transformation between the previous and next frames. These points are discretely distributed and have very low density. They are considered as noise points and removed. P’ 21 d ;

[0096] The removal method is to cluster by Euclidean distance, and get Clusters are formed and a voxel grid is established. If a voxel range contains at least one point, it means that the voxel grid is occupied:

[0097]

[0098]

[0099] in is the number of points in each cluster set; is the number of occupied voxel grids corresponding to each cluster;

[0100] The average cluster density of each cluster set is:

[0101]

[0102] The removal of low-density noise points is done as follows: Medium density is less than part of;

[0103] Using the KD tree structure, extract P’ 21 d and the points in the post-detection point cloud within a certain radius, to intercept the changed parts of the two frames as comprehensively as possible, and obtain the final transformed point cloud P’ 2 d ;

[0104] In the final transformation point cloud P’ 2 d The object recognition algorithm is processed on the transformed part to realize the recognition of all target objects in the point cloud.

[0105] Add the recognized 3D space pose results to the cache list and l The best 3D spatial pose result is selected and sent to the robot controller to perform operations such as grabbing. The remaining 3D spatial pose results remain in the cache list. l middle.

[0106] Continue to acquire the next frame, and repeat the steps of processing the second frame point cloud in sequence until there is no change in the point cloud data in the scene.

[0107] It should be noted that although the above relative transformation point cloud removal method is introduced by taking Euclidean distance clustering as an example, those skilled in the art will appreciate that the present disclosure should not be limited thereto. In fact, the user can flexibly set the removal method according to personal preferences and / or actual application scenarios, as long as the noise is removed.

[0108] In this way, by identifying and acquiring the front frame point cloud and the back frame point cloud of the object to be grasped; downsampling the front frame point cloud and the back frame point cloud data respectively, obtaining the front and back frame sampling point clouds; searching the front and back frame sampling point clouds, and obtaining the relative transformation point clouds from the front and back frame sampling point clouds according to the preset extraction conditions; searching the transformation point cloud and the back frame point cloud, and obtaining the final transformation point cloud from the transformation point cloud and the back frame point cloud according to the preset radius; performing recognition processing on the final transformation point cloud, adding the recognized three-dimensional space pose result to the cache list, selecting the best three-dimensional space pose result in the cache list and sending it to the robot. The front and back frame changes of the point cloud data are effectively utilized, while taking into account the recognition accuracy and speed, and improving the grasping recognition efficiency of the robot arm. At the same time, the problem of unnecessary computational burden caused by redundant recognition and posture screening is solved. Compared with direct recognition on all data of a frame of point cloud, the redundant and time-consuming calculations caused by repeated recognition of relatively static targets in multiple frames have a more efficient recognition speed. At the same time, a caching mechanism for the three-dimensional spatial pose results is introduced to continuously update and select the best grasping pose. This will significantly improve efficiency when the scene contains more target objects.

[0109] Example 2

[0110] Based on the implementation principle and technical means of Example 1, this embodiment provides a device for implementing the above method. The functions and principles of its specific modules are shown in Example 1 and will not be repeated here.

[0111] According to another aspect of the present disclosure, there is also provided a device for implementing the above-mentioned method for recognizing the three-dimensional spatial posture of an object grasped by a robotic arm, comprising a point cloud acquisition module, a downsampling processing module, a relative transformation point cloud acquisition module, a final transformation point cloud acquisition module and a recognition processing module, wherein:

[0112] The point cloud acquisition module is used to identify and acquire the previous frame point cloud and the next frame point cloud of the object to be grasped;

[0113] The point cloud acquisition module uses a three-dimensional camera to obtain the point cloud data of the object to be grasped.

[0114] The downsampling processing module is used to perform downsampling processing on the previous frame point cloud and the next frame point cloud data respectively to obtain the previous and next frame sampling point clouds;

[0115] The relative transformation point cloud acquisition module is used to search the sampling point clouds of the previous and next frames, and obtain the relative transformation point clouds from the sampling point clouds of the previous and next frames according to preset extraction conditions;

[0116] The final transformed point cloud acquisition module is used to search the transformed point cloud and the subsequent frame point cloud, and obtain the final transformed point cloud from the transformed point cloud and the subsequent frame point cloud according to a preset radius;

[0117] The recognition processing module is used to perform recognition processing on the final transformed point cloud, add the recognized three-dimensional space pose result to a cache list, select the best three-dimensional space pose result in the cache list and send it to the robotic arm.

[0118] As a preferred implementation scheme of the present application, optionally, it further includes a first noise reduction module,

[0119] The first noise reduction module is used to preset filtering conditions; and, according to the preset filtering conditions, remove invalid point data in the previous frame point cloud and the next frame point cloud; and, filter to remove outlier noise in the previous frame point cloud and the next frame point cloud.

[0120] As a preferred implementation scheme of the present application, optionally, the relative transformation point cloud acquisition module includes:

[0121] Sampling acquisition module: used to acquire and search the sampling point clouds of the previous and next frames;

[0122] The first extraction module is used to extract the nearest point of the sampling point cloud of the subsequent frame in the sampling point cloud of the previous frame according to the preset extraction conditions;

[0123] The first point collection module is used to collect the nearest point and obtain the relative transformation point cloud of the subsequent frame sampling point cloud relative to the previous frame sampling point cloud.

[0124] As a preferred implementation scheme of the present application, optionally, the final transformed point cloud acquisition module includes:

[0125] A second noise reduction module is used to obtain the relative transformation point cloud, and perform noise reduction processing on the relative transformation point cloud to obtain a noise reduction point cloud;

[0126] A second extraction module: used to obtain and search the noise reduction point cloud and the subsequent frame point cloud, and extract points within the preset radius from the noise reduction point cloud and the subsequent frame point cloud respectively;

[0127] The second point collection module is used to collect points within the preset radius, collect and obtain the final transformed point cloud.

[0128] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0129] Example 3

[0130] Furthermore, according to another aspect of the present disclosure, a control system is provided, comprising:

[0131] processor;

[0132] a memory for storing processor-executable instructions;

[0133] Wherein, the processor is configured to implement the three-dimensional spatial posture recognition method for object grasping by a robotic arm as described in the above-mentioned embodiment 1 when executing the executable instructions.

[0134] The control system of the embodiment of the present disclosure includes a processor and a memory for storing processor executable instructions. The processor is configured to implement any of the above-mentioned methods for recognizing the three-dimensional spatial posture of an object grasped by a robot arm when executing the executable instructions.

[0135] Here, it should be noted that the number of processors can be one or more. At the same time, in the control system of the embodiment of the present disclosure, an input device and an output device may also be included. Among them, the processor, memory, input device and output device may be connected through a bus or in other ways, which are not specifically limited here.

[0136] The memory is a computer-readable storage medium for a method for recognizing the three-dimensional spatial posture of an object grasped by a robot arm, and can be used to store software programs, computer executable programs, and various modules, such as: a program or module corresponding to a method for recognizing the three-dimensional spatial posture of an object grasped by a robot arm in an embodiment of the present disclosure. The processor executes various functional applications and data processing of the control system by running the software programs or modules stored in the memory.

[0137] The input device can be used to receive input numbers or signals. The signal can be a key signal related to user settings and function control of the device / terminal / server. The output device can include a display device such as a display screen.

[0138] Example 4

[0139] According to another aspect of the present disclosure, a non-volatile computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the above-mentioned method for recognizing the three-dimensional spatial posture of an object grasped by a robotic arm is implemented.

[0140] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for recognizing the three-dimensional spatial position of an object grasped by a robotic arm, characterized in that: The steps include: S100, identifying and acquiring the previous frame point cloud and the next frame point cloud of the object to be grasped; S200, down-sampling the previous frame point cloud and the next frame point cloud data respectively to obtain the previous and next frame sampling point clouds; S300, searching for the sampling point clouds of the previous and next frames, and obtaining a relative transformation point cloud from the sampling point clouds of the previous and next frames according to a preset extraction condition; the relative transformation point cloud is the relative change point cloud data of the point cloud of the next frame relative to the point cloud of the previous frame, extracting the nearest point of the point cloud of the next frame among the points of the point cloud of the previous frame, obtaining the part of the point cloud data of the next frame that changes compared with the point cloud data of the previous frame, and taking it as the relative transformation point cloud; S400, searching the relative transformation point cloud and the subsequent frame point cloud, and obtaining a final transformation point cloud from the relative transformation point cloud and the subsequent frame point cloud according to a preset radius; obtaining a final transformation point cloud by respectively extracting points within a preset radius from each point in the subsequent frame point cloud and the relative transformation point cloud and collecting them, thereby capturing the changed parts of the previous and next frames as comprehensively as possible; S500, performing recognition processing on the final transformed point cloud, adding the recognized three-dimensional space pose result to a cache list, selecting the best three-dimensional space pose result in the cache list and sending it to the robotic arm.

2. The method for recognizing the three-dimensional spatial position of an object grasped by a robot arm according to claim 1, characterized in that: In step S100, after identifying and acquiring the previous frame point cloud and the next frame point cloud of the object to be grasped, the following steps are also included: S101, preset filtering conditions; S102, removing invalid point data in the previous frame point cloud and the next frame point cloud according to a preset filtering condition; and S103 , filtering to remove outlier noise in the previous frame point cloud and the next frame point cloud.

3. The method for recognizing the three-dimensional position of an object grasped by a robot arm according to claim 1 or 2, characterized in that: In step S300, searching the sampling point clouds of the previous and next frames, and obtaining relative transformation point clouds from the sampling point clouds of the previous and next frames according to preset extraction conditions, includes: S310, obtaining and searching the sampling point clouds of the previous and next frames; S320, extracting the nearest point of the sampling point cloud of the subsequent frame in the sampling point cloud of the previous frame according to the preset extraction condition; S330 , collecting the nearest point to obtain a relative transformation point cloud of the subsequent frame sampling point cloud relative to the previous frame sampling point cloud.

4. The method for recognizing the three-dimensional spatial position of an object grasped by a robot arm according to claim 3, characterized in that: In step S400, searching the transformed point cloud and the subsequent frame point cloud, and obtaining the final transformed point cloud from the transformed point cloud and the subsequent frame point cloud according to a preset radius, includes: S410, acquiring the relative transformation point cloud, and performing noise reduction processing on the relative transformation point cloud to obtain a noise reduction point cloud; S420, acquiring and searching the noise reduction point cloud and the subsequent frame point cloud, and extracting points within the preset radius from the noise reduction point cloud and the subsequent frame point cloud respectively; S430: Collect points within the preset radius, and collect and obtain the final transformed point cloud.

5. A device for implementing the method for recognizing the three-dimensional spatial position of an object grasped by a robot arm according to any one of claims 1 to 4, characterized in that: It includes a point cloud acquisition module, a downsampling processing module, a relative transformation point cloud acquisition module, a final transformation point cloud acquisition module and a recognition processing module, wherein: The point cloud acquisition module is used to identify and acquire the previous frame point cloud and the next frame point cloud of the object to be grasped; The downsampling processing module is used to perform downsampling processing on the previous frame point cloud and the next frame point cloud data respectively to obtain the previous and next frame sampling point clouds; The relative transformation point cloud acquisition module is used to search for the sampling point clouds of the previous and next frames, and obtain the relative transformation point cloud from the sampling point clouds of the previous and next frames according to the preset extraction conditions; the relative transformation point cloud is the relative change point cloud data of the point cloud of the next frame relative to the point cloud of the previous frame, extract the nearest point of the point cloud of the next frame among the points of the point cloud of the previous frame, and obtain the part of the point cloud data of the next frame that changes compared with the previous frame, which is used as the relative transformation point cloud; The final transformation point cloud acquisition module is used to search the relative transformation point cloud and the subsequent frame point cloud, and obtain the final transformation point cloud from the relative transformation point cloud and the subsequent frame point cloud according to a preset radius; the final transformation point cloud is obtained by respectively extracting points within the preset radius of each point in the subsequent frame point cloud and the relative transformation point cloud and collecting them, so as to intercept the changed parts of the previous and next frames as comprehensively as possible; The recognition processing module is used to perform recognition processing on the final transformed point cloud, add the recognized three-dimensional space pose result to a cache list, select the best three-dimensional space pose result in the cache list and send it to the robotic arm.

6. The device according to claim 5, characterized in that Also includes a first noise reduction module, The first noise reduction module is used to preset filtering conditions; and, according to the preset filtering conditions, remove invalid point data in the previous frame point cloud and the next frame point cloud; and, filter to remove outlier noise in the previous frame point cloud and the next frame point cloud.

7. The device according to claim 5 or 6, characterized in that The relative transformation point cloud acquisition module includes: Sampling acquisition module: used to acquire and search the sampling point clouds of the previous and next frames; The first extraction module is used to extract the nearest point of the sampling point cloud of the subsequent frame in the sampling point cloud of the previous frame according to the preset extraction conditions; The first point collection module is used to collect the nearest point and obtain the relative transformation point cloud of the subsequent frame sampling point cloud relative to the previous frame sampling point cloud.

8. The device according to claim 7, characterized in that The final transformed point cloud acquisition module includes: A second noise reduction module is used to obtain the relative transformation point cloud, and perform noise reduction processing on the relative transformation point cloud to obtain a noise reduction point cloud; A second extraction module: used to obtain and search the noise reduction point cloud and the subsequent frame point cloud, and extract points within the preset radius from the noise reduction point cloud and the subsequent frame point cloud respectively; The second point collection module is used to collect points within the preset radius, collect and obtain the final transformed point cloud.

9. A control system, characterized in that: include: processor; a memory for storing processor-executable instructions; Wherein, the processor is configured to implement the three-dimensional spatial posture recognition method for object grasping by a robotic arm as described in any one of claims 1 to 4 when executing the executable instructions.

10. A non-volatile computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method for recognizing the three-dimensional spatial posture of an object grasped by a robotic arm as described in any one of claims 1 to 4 is implemented.

Images