Mechanical arm repeated grabbing planning method and device, terminal equipment and storage medium

Abstract

The application belongs to the technical field of intelligent machines, and discloses a planning method and device for repeated grabbing of a mechanical arm, a terminal device and a storage medium, wherein the method comprises the following steps: acquiring initial point cloud of a scene where an object to be grabbed is located; taking the initial point cloud as initial input; sequentially processing the initial point cloud through three stages of a pose generation network, a pose adjustment network and a pose discrimination network; generating first stable pose data and second stable pose data layer by layer; simultaneously generating first stable point cloud data, second stable point cloud data and third stable point cloud data layer by layer according to the stable pose data; taking the stable point cloud data as input of next-stage processing; and finally obtaining a grabbing strategy of the mechanical arm according to the third stable point cloud data. According to the initial point cloud of the scene where the object to be grabbed is located, the application can predict multiple stable positions and stable postures of the object to be grabbed in an application scene, and thus realizes planning of repeated grabbing of the mechanical arm.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of intelligent machinery, and particularly relates to a mechanical arm repeated grabbing planning method and device, a terminal equipment and a storage medium. BACKGROUND

[0002] A mechanical arm is a sophisticated bionic structure integrating multiple modules such as perception, planning and execution, and can realize operations such as grabbing, rotating and placing objects, and is widely used in household and industrial production fields. In application scenarios such as part sorting and part processing, due to the geometric properties of the objects and the working space of the mechanical arm, the mechanical arm cannot realize the transformation of the pose of the target object through single grabbing, and often needs to use the mechanical arm to grab and place the target object multiple times.

[0003] However, since the repeated grabbing process of the mechanical arm involves multiple adjustments of the stable position and stable pose of the target object, the prior art is difficult to realize stable repeated grabbing of complex objects by the mechanical arm. SUMMARY

[0004] Therefore, the embodiments of the present application provide a mechanical arm repeated grabbing planning method and device, a terminal equipment and a storage medium, which can predict multiple stable positions and stable poses of a to-be-grabbed object in an application scenario according to an initial point cloud of the to-be-grabbed object in the application scenario, and then realize planning of repeated grabbing of the mechanical arm.

[0005] In a first aspect, the embodiments of the present application provide a mechanical arm repeated grabbing planning method, comprising:

[0006] Obtaining an initial point cloud of a scene where a to-be-grabbed object is located;

[0007] Inputting the initial point cloud into a pose generation network to obtain first stable pose data;

[0008] Obtaining first stable point cloud data according to the first stable pose data and the initial point cloud;

[0009] Inputting the first stable point cloud data into a pose adjustment network to obtain stable tangent plane data;

[0010] Obtaining second stable pose data according to the stable tangent plane data;

[0011] Obtaining second stable point cloud data according to the second stable pose data and the first stable point cloud data;

[0012] Inputting the second stable point cloud into a pose discrimination network to obtain third stable point cloud data;

[0013] Obtaining a grabbing strategy of the mechanical arm according to the third stable point cloud.

[0014] In a second aspect, an embodiment of the present application provides a planning device for repeated grasping of a mechanical arm, comprising:

[0015] An initial point cloud acquisition module is configured to acquire an initial point cloud of a scene in which an object to be grasped is located.

[0016] A first pose acquisition module is configured to input the initial point cloud into a pose generation network to obtain first stable pose data.

[0017] A first point cloud acquisition module is configured to obtain first stable point cloud data according to the first stable pose data and the initial point cloud.

[0018] A tangent plane acquisition module is configured to input the first stable point cloud data into a pose adjustment network to obtain stable tangent plane data.

[0019] A second pose acquisition module is configured to obtain second stable pose data according to the stable tangent plane data.

[0020] A second point cloud acquisition module is configured to obtain second stable point cloud data according to the second stable pose data and the first stable point cloud data.

[0021] A third point cloud acquisition module is configured to input the second stable point cloud data into a pose discrimination network to obtain third stable point cloud data.

[0022] A grasping strategy generation module is configured to obtain a grasping strategy of the mechanical arm according to the third stable point cloud.

[0023] In a third aspect, an embodiment of the present application provides a terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method of the first aspect when executing the computer program.

[0024] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, which stores a computer program, wherein the computer program is executable by a processor to implement the steps of the method of the first aspect of the embodiment of the present application.

[0025] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when executed on a terminal device, causes the terminal device to perform the steps of the method of the first aspect.

[0026] The method for planning repeated grabbing of a mechanical arm provided in the first aspect of the application comprises the following steps: obtaining an initial point cloud of a scene in which an object to be grabbed is located; inputting the initial point cloud into a pose generation network to obtain first stable pose data; obtaining first stable point cloud data according to the first stable pose data and the initial point cloud; inputting the first stable point cloud data into a pose adjustment network to obtain stable tangent plane data; obtaining second stable pose data according to the stable tangent plane data; obtaining second stable point cloud data according to the second stable pose data and the first stable point cloud data; inputting the second stable point cloud data into a pose discrimination network to obtain third stable point cloud data; and obtaining a grabbing strategy of the mechanical arm according to the third stable point cloud data. The application can predict multiple stable positions and stable postures of the object to be grabbed in an application scene according to the initial point cloud of the scene in which the object to be grabbed is located, and thus the planning of repeated grabbing of the mechanical arm is realized.

[0027] It can be understood that the beneficial effects of the second aspect to the fifth aspect can be referred to the related description in the first aspect, which will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0028] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0029] Figure 1 is a process schematic diagram for realizing repeated grabbing of a mechanical arm provided by the embodiments of the present application;

[0030] Figure 2 is a flow schematic diagram of the method for planning repeated grabbing of a mechanical arm provided by the embodiments of the present application;

[0031] Figure 3 is a process schematic diagram for repeatedly grabbing a first object by a mechanical arm provided by the embodiments of the present application;

[0032] Figure 4 is a process schematic diagram for repeatedly grabbing a second object by a mechanical arm provided by the embodiments of the present application;

[0033] Figure 5 is a structural schematic diagram of a planning device for repeated grabbing of a mechanical arm provided by the embodiments of the present application;

[0034] Figure 6 is a structural schematic diagram of a terminal device provided by the embodiments of the present application. DETAILED DESCRIPTION

[0035] In the following description, for purposes of explanation and not limitation, specific details are set forth such as particular architectures, techniques, etc. in order to provide a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application can be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known methods, devices, circuits, and

[0036] It is to be understood that the terminology "includes", "comprises", "consists of", "consists essentially of" and like terms as used in the specification and following claims are intended to be open-ended and to mean that other components, steps, features, integers, and / or combinations thereof are optionally added to the described components, steps, features, integers, and / or combinations thereof.

[0037] It is also to be understood that the terminology "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items, and that the term "at least one of' encompasses one or more of the associated listed items.

[0038] In addition, the terms "first", "second", "third", etc. as used in the description and the appended claims are used for distinguishing between similar elements and not necessarily for describing a sequential or chronological order. It is to be understood that the terms so used are interchangeable under appropriate circumstances such that the embodiments of the application described herein are, for example, capable of use in either order.

[0039] The terms "one embodiment", "an embodiment", "some embodiments", "another embodiment", "certain embodiments", "certain implementations", "some implementations", "one implementation", "some implementations" and the like, as used herein, are used to describe the best mode contemplated for one or more embodiments of the application. Thus, the above terms (phrases) should be construed to cover embodiments of the application where more than one embodiment can exist and where the subject matter described and / or claimed in one embodiment can be combined with the subject matter described and / or claimed in another embodiment. The use of the term "or" as used herein is to be interpreted as inclusive or meaning any one or any combination. Therefore, "A or B" means "A, B, or both A and B". The use of the term "and" as used herein is to be interpreted as both inclusive and exclusive unless otherwise indicated. Therefore, "A and B" means "A and / or B". The use of the term "and / or" as used herein is to be interpreted as both inclusive and exclusive unless otherwise indicated. Therefore, "A and / or B" means "A, B, or both A and B". The use of the term "and / or" as used herein is to be interpreted as both inclusive and exclusive unless otherwise indicated. Therefore, "A and / or B" means "A, B, or both A and B".

[0040] The planning method for repeated grabbing of the mechanical arm provided in the embodiments of the present application can be executed by the processor of the terminal device when running a computer program with corresponding functions, and the initial point cloud of the scene where the object to be grabbed is located is obtained; the initial point cloud is input into a pose generation network to obtain first stable pose data; the first stable pose data and the initial point cloud are used to obtain first stable point cloud data; the first stable point cloud data is input into a pose adjustment network to obtain stable tangent plane data; the second stable pose data is obtained according to the stable tangent plane data; the second stable pose data and the first stable point cloud data are used to obtain second stable point cloud data; the second stable point cloud data is input into a pose discrimination network to obtain third stable point cloud data; and the third stable point cloud data is used to obtain the grabbing strategy of the mechanical arm. The present application can predict multiple stable positions and stable attitudes of the object to be grabbed in the application scene according to the initial point cloud of the scene where the object to be grabbed is located, and then realize the planning of repeated grabbing of the mechanical arm.

[0041] As shown in Figure 1 , an example process diagram for realizing repeated grabbing of a mechanical arm is exemplarily shown.

[0042] In applications, the terminal device can be a tablet personal computer (Tablet PC), a laptop, a personal computer (PC), a (cloud) server, and the like, which can realize data processing functions. The specific type of the terminal device is not limited in the embodiments of the present application.

[0043] As shown in Figure 2 , in one embodiment, the planning method for repeated grabbing of the mechanical arm provided in the embodiments of the present application includes the following steps S101 to S108:

[0044] In step S101, the initial point cloud of the scene where the object to be grabbed is located is obtained, and step S102 is entered.

[0045] In applications, the depth map of the scene where the object to be grabbed is located can be obtained by a depth camera, and the initial point cloud of the scene where the object to be grabbed is located is obtained according to the depth map.

[0046] In applications, the depth map of the scene where the object to be grabbed is located is obtained by a depth camera, and the initial point cloud of the scene where the object to be grabbed is located is calculated in combination with the built-in parameters of the depth camera. The depth camera can include a structured light depth camera, a binocular stereo vision depth camera, a time of flight (TOF) depth camera, and the like, which can be selected according to actual application needs, and the specific type is not limited in the embodiments of the present application.

[0047] In an embodiment, the obtaining of the initial point cloud of the scene where the object to be grabbed is located comprises:

[0048] obtaining a depth map of the scene where the object to be grabbed is located, and obtaining the initial point cloud according to the depth map.

[0049] In an embodiment, the initial point cloud comprises a background point cloud and an object-to-be-grabbed point cloud.

[0050] In application, the initial point cloud can also be preprocessed to obtain a preprocessed initial point cloud. For example, a Random Sample Consensus (RANSAC), region growing, minimum cut, or other segmentation algorithm can be used to identify the background point cloud and the object-to-be-grabbed point cloud in the initial point cloud, and then the object-to-be-grabbed point cloud is subjected to Gaussian filtering and outlier removal to obtain the preprocessed initial point cloud. The preprocessed initial point cloud has a category label and can distinguish the background point cloud and the object-to-be-grabbed point cloud.

[0051] In step S102, the initial point cloud is input into a pose generation network to obtain first stable pose data, and step S103 is entered.

[0052] In application, the preprocessed initial point cloud can be input into the pose generation network to predict multiple stable poses where the object to be grabbed is preliminarily stably placed in the scene, and the first stable pose data is obtained.

[0053] In an embodiment, the pose generation network comprises a first PointNet++ network, a concatenation network, and a first convolutional neural network.

[0054] The first PointNet++ network is configured to extract first features of the initial point cloud.

[0055] The concatenation network is configured to concatenate the first features and Gaussian noise to obtain concatenated features.

[0056] The first convolutional neural network is configured to obtain the first stable pose data according to the concatenated features.

[0057] In application, the loss function of the pose generation network can be:

[0058]

[0059] wherein, is an approximate geodesic distance, which can be obtained by polynomial expansion of the geodesic distance d geo (R g , R T ).

[0060]

[0061]

[0062] wherein S g represents a set of rotation matrices of all points of the predicted point cloud, S T represents a set of rotation matrices of all points of the real point cloud, R g represents a rotation matrix of any point in the predicted point cloud, R T represents a rotation matrix of any point in the real point cloud, represents R T , the inverse matrix of R represents the trace of R , a i represents the coefficient of the i-th order polynomial, tr i represents the i-th power of R , a is a correction coefficient, which can ensure that the function d geo (R g , R T ) is derivable everywhere.

[0063] In an embodiment, the first stable pose data is 6-DOF data of the object to be grasped in different stable states.

[0064] In an application, the object to be grasped has multiple different stable states in the scene, and each stable state has corresponding 6-DOF data, which are translational degrees of freedom along the x, y, and z axes and rotational degrees of freedom around the x, y, and z axes, respectively.

[0065] Step S103: obtaining first stable point cloud data according to the first stable pose data and the initial point cloud, and entering step S104.

[0066] In an application, the object point cloud in the initial point cloud can be adjusted according to the first stable pose data to obtain multiple first stable point clouds in which the object to be grasped is preliminarily stably placed in the scene, i.e., to obtain the first stable point cloud data.

[0067] Step S104: inputting the first stable point cloud data into a pose adjustment network to obtain stable tangent plane data, and entering step S105.

[0068] In an application, the first stable point cloud data can be input into the pose generation network to predict multiple tangent planes corresponding to multiple stable placement positions of the object to be grasped, and to obtain the stable tangent plane data.

[0069] In an embodiment, the pose adjustment network comprises a second PointNet++ network and a second convolutional neural network.

[0070] The second PointNet++ network is used to extract second features of the first stable point cloud data.

[0071] The second convolutional neural network is used to obtain the stable tangent plane data according to the second features.

[0072] In application, an existing PointNet++ network can be called as the second PointNet++ network.

[0073] In application, the first stable point cloud data can be input into the second PointNet++ network to obtain the second features, and the second features can be input into the second convolutional neural network to obtain the stable tangent plane data.

[0074] In step S105, second stable pose data is obtained according to the stable tangent plane data, and step S106 is entered.

[0075] In application, a plurality of stable poses of the object to be grasped in further stable placement in the scene can be solved according to a functional expression of the stable tangent plane data, and the second stable pose data is obtained.

[0076] In one embodiment, the second stable pose data is 6-DOF data of the object to be grasped in different stable states.

[0077] In step S106, second stable point cloud data is obtained according to the second stable pose data and the first stable point cloud, and step S107 is entered.

[0078] In application, a plurality of first stable point clouds in the first stable point cloud data can be adjusted according to the second stable pose data, and the second stable point cloud data containing a plurality of second stable point clouds is obtained.

[0079] In step S107, third stable point cloud data is obtained by inputting the second stable point cloud data into a pose discrimination network, and step S108 is entered.

[0080] In application, the second stable point cloud data can be input into the pose discrimination network to predict the stability of each second stable point cloud in the second stable point cloud data, remove the second stable point cloud with unstable discrimination result, and screen the second stable point cloud with stable discrimination result, and the third stable point cloud data is obtained.

[0081] In one embodiment, the pose discrimination network includes a third PointNet++ network, a mapping network, a U-Net network and a residual network.

[0082] The third PointNet++ network is used to extract third features of the second stable point cloud data.

[0083] The mapping network is configured to map the third feature to a plane to obtain a plane feature.

[0084] The U-Net network and the residual network are configured to obtain the third stable point cloud data according to the plane feature.

[0085] In application, an existing PointNet++ network can be called as the third PointNet++ network.

[0086] In application, the second stable point cloud data is input into the third PointNet++ network to obtain a third feature, the third feature is input into the mapping network to obtain a plane feature, the plane feature is input into the U-Net and the residual network to obtain a classification result of stability discrimination, which has two types of stable and unstable, and the second stable point cloud data with the classification result of stable is screened out to obtain the third stable point cloud data.

[0087] In step S108, a grasping strategy of the robot arm is obtained according to the third stable point cloud data.

[0088] In application, the UniGrasp network can be used to predict the grasping point of the robot arm according to the third stable point cloud data, and then the motion trajectory of the robot arm is generated to obtain the grasping strategy of the robot arm, so as to realize the planning of repeated grasping of the robot arm. Figure 3 、 Figure 4 As shown in the process schematic diagram of the robot arm repeatedly grasping the first object and the process schematic diagram of the robot arm repeatedly grasping the second object.

[0089] In application, the grasping strategy can be stored in the local storage space of the terminal device, or the grasping strategy can be uploaded to the database of the blockchain or the cloud server for storage, so that users in need can download and use it from the blockchain or the cloud server by using any network device.

[0090] The embodiment of the application further provides a planning device for repeated grasping of a robot arm, which is used to execute the steps in the above-mentioned planning method for repeated grasping of a robot arm. The device can be a virtual appliance in a terminal device, which is run by a processor of the terminal device, or can be the terminal device itself.

[0091] As shown in Figure 5 The planning device for repeated grasping of a robot arm 100 provided by the embodiment of the application comprises:

[0092] An initial point cloud acquisition module 101 is configured to acquire an initial point cloud of a scene where an object to be grasped is located, and enter a first pose acquisition module 102.

[0093] The first pose obtaining module 102 is configured to input the initial point cloud into a pose generation network to obtain first stable pose data, and enter the first point cloud obtaining module 103.

[0094] The first point cloud obtaining module 103 is configured to obtain first stable point cloud data according to the first stable pose data and the initial point cloud, and enter the tangent plane obtaining module 104.

[0095] The tangent plane obtaining module 104 is configured to input the first stable point cloud data into a pose adjustment network to obtain stable tangent plane data, and enter the second pose obtaining module 105.

[0096] The second pose obtaining module 105 is configured to obtain second stable pose data according to the stable tangent plane data, and enter the second point cloud obtaining module 106.

[0097] The second point cloud obtaining module 106 is configured to obtain second stable point cloud data according to the second stable pose data and the first stable point cloud data, and enter the third point cloud obtaining module 107.

[0098] The third point cloud obtaining module 107 is configured to input the second stable point cloud data into a pose discrimination network to obtain third stable point cloud data, and enter the grasping strategy generation module 108.

[0099] The grasping strategy generation module 108 is configured to obtain a grasping strategy of the robot arm according to the third stable point cloud.

[0100] In applications, each unit in the above device can be a software program module, or can be realized by different logical circuits integrated in the processor or independent physical components connected with the processor, or can be realized by multiple distributed processors.

[0101] As shown in Figure 6 The embodiment of the application also provides a terminal device 200, which comprises at least one processor 201 (only one is shown in the figure), a memory 202, and a computer program 203 stored in the memory 202 and executable on the at least one processor 201, and the processor 201 implements the steps in the above-mentioned various robot arm repeated grasping planning method embodiments when executing the computer program 203.

[0102] In applications, the terminal device can include, but is not limited to, a processor and a memory. Those skilled in the art can understand that Figure 6 is only an example of the terminal device and does not constitute a limitation on the terminal device, and can include more or fewer components than shown, or combine certain components, or different components, for example, can also include input / output devices, network access devices, etc.

[0103] In applications, the processor can be a central processing module (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or can also be any conventional processor.

[0104] In applications, the memory can be an internal storage module of the terminal device in some embodiments, for example, a hard disk or a memory of the terminal device. The memory can also be an external storage device of the terminal device in other embodiments, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Further, the memory can include both the internal storage module and the external storage device of the terminal device. The memory is used to store an operating system, application programs, a boot loader, data, and other programs, for example, program codes of the computer programs, etc. The memory can also be used to temporarily store data that has been output or will be output.

[0105] It should be understood that the size of the serial number of each step in the above embodiments does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0106] It should be noted that the information interaction, execution process, etc. between the above devices / units, since based on the same concept as the method embodiments of the present application, the specific functions and the technical effects brought by it can be referred to the method embodiments part, and will not be repeated here.

[0107] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of each functional unit and module is exemplified, and in actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or software functional unit. In addition, the specific names of each functional unit and module are only for easy distinction, and do not limit the protection scope of the present application. The specific working process of the unit and module in the above system can refer to the corresponding process in the foregoing method embodiment, which will not be described here.

[0108] The embodiment of the present application also provides a computer readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps in the above-mentioned various mechanical arm repeated grabbing planning method embodiments can be implemented.

[0109] The embodiment of the present application provides a computer program product. When the computer program product runs on a terminal device, the terminal device can execute the steps in the above-mentioned various mechanical arm repeated grabbing planning method embodiments.

[0110] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on this understanding, the present application can implement all or part of the processes in the above-mentioned embodiment methods by a computer program to instruct related hardware to complete, and the computer program can be stored in a computer readable storage medium. The computer program can implement the steps in the above-mentioned various method embodiments when executed by a processor. The computer program includes computer program code, which can be in the form of source code, object code, executable files or some intermediate forms. The computer readable medium at least includes any entity or device capable of carrying the computer program code to the photographing device / test equipment, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium, such as U disk, mobile hard disk, magnetic disk or optical disk, etc. In some jurisdictions, according to legislation and patent practice, the computer readable medium cannot be an electrical carrier signal and a telecommunication signal.

[0111] In the above embodiments, the description of each embodiment focuses on different aspects, and the parts not described or recorded in a certain embodiment can be referred to the relevant description of other embodiments.

[0112] Those skilled in the art can understand that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. The skilled person can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0113] In the embodiments provided in the present application, it should be understood that the disclosed apparatus / network device and method can be implemented in other ways. For example, the apparatus / network device embodiments described above are merely schematic, for example, the division of the modules or units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed units can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0114] The units described as separate components can or can not be physically separated, and the components displayed as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiments.

[0115] The above described embodiments are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A method for planning of mechanical arm repeated grasping, characterized in that, The method comprises: obtaining an initial point cloud of a scene where an object to be grabbed is located; inputting the initial point cloud into a pose generation network to obtain first stable pose data; obtaining first stable point cloud data according to the first stable pose data and the initial point cloud; inputting the first stable point cloud data into a pose adjustment network to obtain stable tangent plane data; obtaining second stable pose data according to the stable tangent plane data; obtaining second stable point cloud data according to the second stable pose data and the first stable point cloud data; inputting the second stable point cloud data into a pose discrimination network to obtain third stable point cloud data; obtaining a grabbing strategy of a mechanical arm according to the third stable point cloud data.

2. The method of claim 1, wherein, The method comprises: obtaining a depth map of a scene where an object to be grabbed is located, and obtaining the initial point cloud according to the depth map.

3. The method of claim 2, wherein, The initial point cloud comprises background point cloud and object-to-be-grabbed point cloud.

4. The method of claim 1, wherein, The pose generation network comprises a first PointNet++ network, a concatenation network and a first convolutional neural network; the first PointNet++ network is configured to extract first features of the initial point cloud; the concatenation network is configured to concatenate the first features and Gaussian noise to obtain concatenated features; the first convolutional neural network is configured to obtain the first stable pose data according to the concatenated features.

5. The method of claim 1, wherein, The pose adjustment network comprises a second PointNet++ network and a second convolutional neural network; the second PointNet++ network is configured to extract second features of the first stable point cloud data; the second convolutional neural network is configured to obtain the stable tangent plane data according to the second features.

6. The method of claim 1, wherein, The pose discrimination network comprises a third PointNet++ network, a mapping network, a U-Net network and a residual network; the third PointNet++ network is configured to extract third features of the second stable point cloud data; the mapping network is configured to map the third features to a plane to obtain plane features; the U-Net network and the residual network are configured to obtain the third stable point cloud data according to the plane features.

7. The method of claim 1, wherein, The first stable pose data and the second stable pose data are six-degree-of-freedom data of the object to be grabbed in different stable states. 8.A device for planning of mechanical arm repeated grasping, characterized in that, The method comprises: an initial point cloud obtaining module configured to obtain an initial point cloud of a scene where an object to be grabbed is located; a first pose obtaining module configured to input the initial point cloud into a pose generation network to obtain first stable pose data; a first point cloud obtaining module configured to obtain first stable point cloud data according to the first stable pose data and the initial point cloud; a tangent plane obtaining module configured to input the first stable point cloud data into a pose adjustment network to obtain stable tangent plane data; a second pose obtaining module configured to obtain second stable pose data according to the stable tangent plane data; a second point cloud obtaining module configured to obtain second stable point cloud data according to the second stable pose data and the first stable point cloud data; A third point cloud acquisition module is configured to input the second stable point cloud data into a pose discrimination network to obtain third stable point cloud data. A grabbing strategy generation module is configured to obtain a grabbing strategy of the mechanical arm according to the third stable point cloud data.

9. A terminal device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the steps of the method of any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 9. The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 7.

Images