Autonomous intelligent decision-making method for capturable position of spatial non-cooperative target

Abstract

The invention belongs to the technical field of aerospace, and particularly relates to an autonomous intelligent decision-making method for a capturable position of a spatial non-cooperative target. The method comprises the steps of 1, generating a space target point cloud; step 2, constructing a capture prediction network; step 3, training the capture prediction network constructed in the step 2; and step 4, utilizing the capture prediction network trained in the step 3 to perform prediction capture on the space target point cloud generated in the step 1. The method is used for solving the computer vision problem of grabbing prediction.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of aerospace, and particularly relates to a method for autonomous intelligent decision of a capturable position of a space non-cooperative target. BACKGROUND

[0002] With the increasing human space activities, the number of satellites in orbit is increasing year by year, and the demand for on-orbit services is also growing, while the complexity of the task is gradually increasing. According to the statistical data, in 2010, there were 401 satellites in orbit in normal operation, and 183 failed satellites, excluding the satellites with unknown failure causes, of which 88 failed satellites could be repaired by on-orbit services. According to the analysis of the statistical data of the operation status of the on-orbit spacecraft, a considerable part of the failed spacecraft is repairable. In order to extend the service life and value of the spacecraft and ensure the safety of the on-orbit spacecraft in normal operation, on-orbit repair of failed spacecraft is an urgent task to be completed.

[0003] The technology for achieving stable control of the target spacecraft has been widely used in various on-orbit services, such as on-orbit repair of failed satellites, space on-orbit assembly, and orbit debris cleaning. Such tasks have the characteristics of precise positioning and dependence on grabbing. The traditional method for grabbing the on-orbit satellite requires astronauts or ground operators to control the mechanical arm. This task mode often requires long-term training of the operators, and has high task cost, low grabbing efficiency, and difficulty in ensuring the grabbing quality of long-time operation. Therefore, the non-cooperative target spacecraft capture requiring the intervention of astronauts is more suitable for scenarios with small task quantity and short operation time, and it is difficult to meet the increasing demand for on-orbit services and the increasing complexity of the tasks. Compared with the operation mode requiring the intervention of astronauts for assistance or guidance, the use of deep learning network for satellite grabbing prediction has the advantages of high recognition accuracy and high efficiency, and can better meet the increasing demand for on-orbit services and the increasing complexity of the tasks. SUMMARY

[0004] The application provides a method for autonomous intelligent decision of a capturable position of a space non-cooperative target, which is used to solve the computer vision problem of grabbing prediction.

[0005] The application is implemented by the following technical solutions:

[0006] A method for autonomous intelligent decision of a capturable position of a space non-cooperative target, the generation method comprising the following steps,

[0007] Step 1: generating a space target point cloud;

[0008] Step 2: constructing a grabbing prediction network;

[0009] Step 3: Train the crawling prediction network constructed in Step 2;

[0010] Step 4: Use the grasping prediction network trained in Step 3 to predict and grasp the spatial target point cloud generated in Step 1.

[0011] Furthermore, step 1 specifically involves assuming the pixel coordinates P of the color image are... rgb Represented as (u R ,v R ,z R ) T , where u R v R , z R Let p represent the x-coordinate, y-coordinate, and depth value in the camera coordinate system of the color image, respectively, and let p be the projection of the image onto the color image plane. rgb ; Pixel coordinates P of the depth image ir It can be represented as (u L ,v L ,z L ) T Let represent the x-coordinate, y-coordinate, and depth value in the camera coordinate system of the depth image, respectively, and let p be the projection of the image onto the image plane. ir From the pinhole imaging model of the camera, we can obtain:

[0012]

[0013] P rgb With P ir Representing the spatial coordinates of the same point in different camera coordinate systems, the transformation relationship between two different coordinate systems can be described by a rotation matrix R and a translation matrix T, that is:

[0014] P rgb =RP ir +T (2)

[0015] The rotation and translation matrices are obtained from the extrinsic parameters of the camera calibration. After coordinate transformation, the color image and the depth image are registered, and an RGB-D image is obtained.

[0016] Based on the camera's intrinsic parameter matrix and the pixel coordinates (u, v) provided by the color image in the pixel coordinate system, the coordinates (x, y) of the pixel in the camera coordinate system can be calculated. The depth image can directly provide the pixel depth z in the camera coordinate system. Therefore, the three-dimensional coordinates P = (x, y, z) in the camera coordinate system can be obtained. The camera coordinate system P and the pixel coordinate system P... uv The relationship between the points below can be represented as:

[0017]

[0018] wherein H represents an intrinsic matrix of the camera, f x is a focal length of the camera along the x-axis, y is a focal length of the camera along the y-axis, x and c y are the x-coordinate and y-coordinate of the base point, respectively, and c is the center along the x-axis and y-axis of the image plane, and z=d is the depth value measured by the camera;

[0019] According to the homogeneous transformation matrix T from the world coordinate system to the pixel coordinate system, the following can be obtained:

[0020]

[0021] wherein R is a rotation matrix from the world coordinate system to the pixel coordinate system, and t is a corresponding translation; thus, the point cloud coordinates in the world coordinate system can be obtained.

[0022] As can be seen from the figure, the obtained point cloud image is relatively complete and has no obvious point cloud void, and can be used for grasping prediction.

[0023] Further, the step 2 specifically comprises the following steps,

[0024] Step 2.1: defining a two-finger gripper robot grasping pose and its coordinate system;

[0025] Step 2.2: constructing a grasping prediction network based on the grasping pose and the coordinate system of step 2.1.

[0026] Further, the step 2.1 is specifically that the grasping pose is recorded as G, and is defined in detail as:

[0027] G=[R tω] T (5)

[0028] wherein R∈R 3×3 represents the direction of the gripper; t∈R 3×1 represents the position of the center of the gripper when reaching the grasping pose; and ω∈R represents the width that should be opened when the gripper grasps the target.

[0029] By predicting a feasible grasping point, based on this point, a vector that can approach the point, the distance of the gripper when clamping, and a gripper width that can clamp the target are predicted, and based on the predicted vector, a rotation in the plane along the vector direction is obtained.

[0030] The origin of the coordinate system is the center of the two-finger gripper, and the z-axis direction is the same as the approaching vector direction.

[0031] Further, the grasping prediction network of step 2.2 comprises an approaching network, an operation network and a tolerance network, the approaching network uses PointNet++ as the core network, which is used for viewpoint classification and extraction of point cloud features;

[0032] Taking N three-dimensional points as input, a new point set with C-dimensional features can be realized by calling the encoder-decoder module of PointNet++, and the point cloud in the whole scene is down-sampled using the farthest point sampling method mentioned above, and then M points are output. The output of the proximity network is a vector of size M x (2+V), where 2 stores the binary category representing graspable or ungraspable, and V represents the number of predefined proximity vectors;

[0033] The proximity network loss function is specifically defined as a binary label indicating whether it is graspable. For each predicted grasp point, a binary label is assigned. For these predicted points, the ball query method in PointNet++ is used to construct a spherical space with a radius of r with the point as the origin. If there is at least one real point that can be grasped in this spherical space, the point is considered as a graspable point, and its binary label is assigned as 1. Then, for each graspable point, V virtual proximity vectors are sampled under the camera. The i-th grasp point's j-th virtual view proximity vector is denoted as v ij The objective function can be obtained as follows:

[0034]

[0035] where c i represents the prediction of whether the point is graspable. If it is positive, c is 1, and if it is negative, it is assigned a value of 0. s ij represents the prediction confidence score of point i from view j, is its corresponding real situation, represents the angle difference, which is limited to within 5° by the indicator function 1(), and for L cls a two-class softmax loss function is used, as shown in equation (7), for L reg To avoid the problem of loss difficulty to converge and out-of-distribution gradient value explosion, a smooth L1 loss is used as shown in equation (8):

[0036]

[0037] where y is the sample label, is the probability that the model predicts the sample to be positive. If the sample is positive, it takes a value of 0, otherwise it takes a negative value,

[0038]

[0039] where β is set to 1;

[0040] the proximity vector vij After that, all the grasp confidences are in the interval of (0, 1] and are divided into 10 classes, the closer to 0 represents the lower score, and vice versa; before inputting the feasible grasp candidates into the operation network, a unified representation is established for all the grasp candidates, which is converted into a (3+C) dimensional vector; the approaching distance is divided into K intervals, then each distance can be represented as d K ; for a given d K , a cylinder is constructed with the approaching vector center, and a fixed number of sampling points are sampled in the cylinder; a new coordinate system is defined, with the grasp point as the origin, v ij as the z-axis, all the sampling points are converted into the new coordinate system, and the conversion matrix O ij is calculated as follows:

[0041]

[0042] wherein, and represents the 2nd element of v ij , and so on.

[0043] Further, for the rotation in the plane, the effect of using the classification method for prediction is better than that of regression; taking the aligned point cloud as the input, a classification score and a normalized residual error are predicted for each rotation, as well as the corresponding clamping width and prediction confidence; since the gripper is axisymmetric in the geometric plane, only the rotation angle in [0, 180°) is predicted; for the design of the objective function, it is as follows:

[0044]

[0045] wherein R ij represents the rotation degree, S ij , W ij and d respectively represent the confidence score, the gripper width and the approaching distance. L d represents the loss of the dth distance.

[0046] Further, the tolerance network is specifically, for a given real grasp pose, searching for its neighborhood in the space to find the maximum distance at which the grasp is still robust and the grasp score is greater than 0.5, and setting this behavior as the objective function;

[0047] The loss function can be written as:

[0048]

[0049] wherein, T ij represents the maximum disturbance that the grasp pose can resist.

[0050] Further, the step 3 comprises data set preprocessing and training of the network;

[0051] The data set preprocessing specifically comprises: first, obtaining a depth map and a color map of different perspectives of a space target based on Blender to obtain point cloud information, sampling to obtain a grasping point from the point cloud, then sampling and evaluating a grasping pose and a gripper grasping depth; finally, projecting the grasping pose to the scene by using the 6D pose of each object;

[0052] The 3D model and data used are put into the GraspNet-1Billion data set, and the data set is arranged into a structure that can be imported;

[0053] Finally, 200 scenes of the data set are divided, the first 100 scenes are used as a training set, and the last 100 scenes are used as an evaluation set.

[0054] Further, the training of the network specifically comprises:

[0055] After obtaining the total objective function of the network, an optimizer is selected; the working mechanism of the optimizer in deep learning is as follows: the optimizer mainly plays a role in the back propagation of training, guiding each parameter of the objective function to update in the direction of gradient descent, so that the updated objective function gradually approaches the global minimum; for the optimizer, there are usually the following input parameters: objective function L, initial learning rate α, iteration number t and parameters to be optimized w; the basic steps of the optimizer updating parameters are as follows:

[0056] 1) Calculate the gradient of the objective function with respect to the current parameters:

[0057] grad t =▽L(w t ) (12)

[0058] 2) Calculate the first-order momentum and the second-order momentum according to the historical gradient:

[0059]

[0060] 3) Calculate the descending gradient at the current time:

[0061]

[0062] 4) Update the Adam optimizer parameters according to the descending gradient:

[0063]

[0064] m t and V t are not the non-difference estimates of the first-order moment and the second-order moment of the gradient, when the training number is small, the mt and V t Start from 0; m t and V t Corrected to and As shown in formula (16):

[0065]

[0066] The Adam parameter update formula is Wherein, epsilon is a small amount, preventing the divisor from being 0.

[0067] A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the method as described above when executing the computer program.

[0068] The beneficial effects of the present application are:

[0069] 1. The present application realizes the automatic capture strategy generation of space non-cooperative targets through deep learning algorithm and target point cloud reconstruction technology, reduces the dependence on artificial intervention, and improves the autonomy and intelligent degree of the task.

[0070] 2. The present application can quickly and accurately identify and generate the best grabbing posture and strategy of non-cooperative targets through the training of the grabbing detection network, greatly improving the operation efficiency and success rate of the space manipulator.

[0071] 3. The present application can adapt to space targets of different shapes and materials, and maintain high robustness and stability in variable space environment, ensuring reliable execution of the task. BRIEF DESCRIPTION OF DRAWINGS

[0072] Figure 1 is the method flowchart of the present application.

[0073] Figure 2 is the target satellite 3D model schematic diagram of the present application.

[0074] Figure 3 is the Blender target image schematic diagram of the present application.

[0075] Figure 4 is the color image and depth image obtained by simulation of the present application, wherein (a) is the color image, and (b) is the depth image.

[0076] Figure 5 is the point cloud schematic diagram obtained after processing of the present application.

[0077] Figure 6is the rotation matrix decoupling diagram of the present application, where the left is the approaching vector diagram, and the right is the in-plane rotation diagram.

[0078] Figure 7 is the coordinate definition diagram of the gripper of the present application.

[0079] Figure 8 is the grasp prediction network structure diagram of the present application.

[0080] Figure 9 is the grasp dataset diagram of the present application, where (a) is a target star A as a whole, (b) is a docking ring of a satellite on a target star B, (c) is a target star C as a whole, and (d) is a sailboard on a target star C.

[0081] Figure 10 is the loss function curve diagram during training of the present application, where (a) is the total loss curve diagram of the network, (b) is the grasp classification loss curve diagram, (c) is the view classification loss curve diagram, (d) is the grasp angle prediction loss curve diagram, (e) is the grasp confidence loss curve diagram, and (f) is the tolerance loss curve diagram.

[0082] Figure 11 is the prediction accuracy curve diagram of the model of the present application during training.

[0083] Figure 12 is the two-fingered gripper diagram for grasp evaluation test of the present application.

[0084] Figure 13 is the satellite docking ring grasping diagram of the present application, where (a) is the diagram of screening graspable docking rings in the satellite docking ring, and (b) is the diagram of the mechanical arm grasping the screened satellite docking ring.

[0085] Figure 14 is the satellite sailboard partial grasping diagram of the present application, where (a) is the diagram of the graspable docking ring of the satellite sailboard, and (b) is the diagram of the mechanical arm grasping the graspable docking ring of the satellite sailboard.

[0086] Figure 15 is the satellite nozzle partial grasping diagram of the present application, where (a) is the diagram of the graspable part of the satellite nozzle, and (b) is the diagram of the mechanical arm grasping the graspable part of the satellite nozzle.

[0087] Figure 16 is the point cloud diagram of different point quantities of the present application, where (a) is the 6×10 4 point cloud, (b) is the 2×10 4 point cloud, and (c) is the 5×10 3 point cloud.

[0088] Figure 17 is a schematic diagram of prediction accuracy of the same number of points of the present application. DETAILED DESCRIPTION

[0089] In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular sequences of steps, techniques, etc., in order to provide a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application can be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.

[0090] It should be understood that the term "comprising" when used in this specification and the appended claims, indicates the presence of stated features, integers, steps, operations, elements, and / or components but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0091] It should also be understood that the terminology used in the description of the present application herein is for the purpose of describing particular embodiments only and is not intended to be limiting. As used in this specification and the appended claims, the singular forms "a," "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

[0092] The following description will be made in conjunction with the accompanying drawings attached to the present application specification Figures 1-17 The technical solutions in the embodiments of the present application are described clearly and completely. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0093] In the following description, many specific details are set forth in order to provide a thorough understanding of the present application. However, the present application can be practiced without the specific details. In other instances, well-known methods, procedures and components have not been described in detail so as not to obscure the present application.

[0094] Embodiment one

[0095] The embodiment obtains a spatial non-cooperative target capture position autonomous intelligent decision method. The key of the capture is to predict a real and reliable capture pose for a given visual input (image or point cloud), and use the predicted capture for high-level tasks to interact with the object. The generation method comprises the following steps,

[0096] Step 1: generate a spatial target point cloud;

[0097] Step 2: constructing a grasping prediction network;

[0098] Step 3: training the grasping prediction network constructed in step 2;

[0099] Step 4: using the grasping prediction network trained in step 3 to predict grasping for the spatial target point cloud generated in step 1.

[0100] Further, the step 1 is specifically, the point cloud, that is, the set of points, is the main processing object in the 3D vision task, and the point cloud has its irreplaceable advantage-depth, that is, the three-dimensional point cloud directly provides the data of the three-dimensional space. But the acquisition of point cloud data depends on various depth sensors such as RGB-D camera or three-dimensional scanner, or is calculated from multi-view color pictures, such as binocular camera. The present application uses an RGB-D camera. The point cloud can be obtained by the RGB-D image collected by the camera, and the point coordinates in the real world are created by changing the coordinate system through the camera parameters. In the field of image processing, the point cloud image is usually obtained by combining color image and depth image, that is, the point cloud is converted based on the RGB-D image; in order to more accurately obtain the depth information of the target and the geometric characteristics of the target satellite, the target satellite point cloud generation is needed.

[0101] Generally, the world coordinates of the depth camera and the color camera do not coincide, so for the depth camera, the color image and the depth image need to be registered when used. The purpose of registration is to convert the image coordinate system of the depth image to the image coordinate system of the color image.

[0102] Suppose the pixel coordinates of the color image P rgb is represented as (u R ,v R ,z R ) T , where u R , v R , z R respectively represent the horizontal coordinate, vertical coordinate and depth value in the camera coordinate system of the color image, and the projection of p rgb on the color image plane is recorded as p ir ; similarly, the pixel coordinates of the depth image P L can be represented as (u L ,v L ,z T ) ir , which respectively represent the horizontal coordinate, vertical coordinate and depth value in the camera coordinate system of the depth image, and the projection of p ir on the image plane is recorded as p ir ; from the pinhole imaging model of the camera, we have:

[0103]

[0104] P rgb With P ir Representing the spatial coordinates of the same point in different camera coordinate systems, the transformation relationship between two different coordinate systems can be described by a rotation matrix R and a translation matrix T, that is:

[0105] P rgb =RP ir +T (2)

[0106] The rotation and translation matrices are obtained from the extrinsic parameters of the camera calibration. After coordinate transformation, the color image and the depth image are registered, and an RGB-D image is obtained.

[0107] An RGB-D image has four channels. The first three channels are RGB, representing the three primary color channels: red, green, and blue. Each channel is represented by an integer between 0 and 255, which corresponds to the color intensity. D represents the depth channel, and the value of this channel provides distance information of the image.

[0108] Based on the camera's intrinsic parameter matrix and the pixel coordinates (u, v) provided by the color image in the pixel coordinate system, the coordinates (x, y) of the pixel in the camera coordinate system can be calculated. The depth image can directly provide the pixel depth z in the camera coordinate system. Therefore, the three-dimensional coordinates P = (x, y, z) in the camera coordinate system can be obtained. The camera coordinate system P and the pixel coordinate system P... uv The relationship between the points below can be represented as:

[0109]

[0110] Where H represents the camera's intrinsic parameter matrix, f x f is the focal length of the camera along the x-axis. y Let c be the focal length of the camera along the y-axis. x and c y These are the x and y coordinates of the base point, respectively, and are the centers along the x and y axes of the image plane. z = d is the depth value measured by the camera.

[0111] Based on the homogeneous transformation matrix T from the world coordinate system to the pixel coordinate system, we can obtain:

[0112]

[0113] Where R is the rotation matrix from the world coordinate system to the pixel coordinate system, and t is the corresponding translation amount; thus, the point cloud coordinates in the world coordinate system can be obtained, and the point cloud image can be constructed based on this.

[0114] Import the satellite's URDF model into Blender and set the depth camera parameters. The resulting image after starting the simulation is as follows. Figure 3As shown:

[0115] The color image and the depth image obtained by the depth camera are shown below (both the color image and the depth image filter the background), the left is the color image, and the right is the depth image (the depth image is converted into a gray image);

[0116] The RGB-D image is obtained after the color image and the depth image are registered, and the point cloud of the satellite can be converted from the RGB-D image as shown. Figure 5

[0117] As can be seen from Figure 5 the above, the obtained point cloud image is relatively complete and has no obvious point cloud void, and can be used for grasp prediction.

[0118] Further, the step 2 specifically includes the following steps,

[0119] Step 2.1: defining a two-finger gripper robot grasp pose and its coordinate system;

[0120] Step 2.2: constructing a grasp prediction network based on the grasp pose and the coordinate system of step 2.1.

[0121] Specifically, the 3D point cloud classification and semantic segmentation network PointNet:

[0122] Point cloud is an important type of geometric data structure, but due to the diversity and relative instability of the file format of point cloud, most scholars will convert the point cloud into a 3D voxel model or a multi-view picture set. However, due to the large size of point cloud data, the above method often has problems such as large memory consumption and low computing efficiency, and even may cause data distortion in the quantization process. Therefore, Charles R. Qi and Hao Su et al. proposed a neural network PointNet based on deep learning, which only uses point cloud as input and can perform deep learning on the features of point cloud, providing a simple and efficient feature processor for point cloud classification and segmentation

[14] .

[0123] Direct deep learning on the features of point cloud needs to consider the characteristics of point cloud: the unordered nature of point cloud, the data points of point cloud have no fixed order; the points of point cloud are not isolated and need to consider local structure; affine independence. Under the consideration of the characteristics of point cloud, PointNet directly takes point cloud as input and classifies and segments the point cloud. The default input point cloud {P i |i=1,...,n} of the network is three-dimensional, and each point P i can be represented by a three-dimensional coordinate vector (x, y, z).

[0124] ​The network can be divided into two parts in structure, the network for classification task and the network for segmentation task, and the classification and segmentation network share a large part of the network structure.

[0125] In the PointNet network, the classification network and the segmentation network use the same part of the framework, and use multi-layer perceptron to fit the nonlinear function, and the specific structure is explained as follows:

[0126] 1) The network for classification task takes n points as input, which enters the transformation network (T-Net) for twice transformation, the main operation is normalization and point translation rotation, etc., to ensure the invariance of the input point cloud, and then output the classification scores of each category.

[0127] 2) The network for segmentation task can be regarded as an extension of the classification network, and the input of the segmentation network is the feature of the point in the classification network and the global feature, and the two feature tensors are fused to obtain a new tensor and output the score of each point.

[0128] 3) "mlp" is a multi-layer perceptron, and the numbers in the parentheses are the sizes of the layers. All layers using the activation function ReLU use the BatchNorm method to normalize the data. A Dropout layer is used in the last multi-layer perceptron of the classification network to prevent overfitting.

[0129] However, PointNet only uses multi-layer perceptron and maximum pooling layer, however, PointNet only uses multi-layer perceptron and maximum pooling network, without the link of regional feature extraction, so the generalization ability and detail processing ability of the model in complex environment are limited. The main problems of PointNet can be summarized as follows:

[0130] 1) The multi-layer perceptron used by PointNet only extracts features for each point, and the ability to integrate local structure information of point cloud is too weak;

[0131] 2) The global feature is directly obtained by the maximum pooling layer, which will cause the loss of point cloud information for classification and semantic segmentation tasks, resulting in lower final score;

[0132] 3) The input in the segmentation network is obtained by copying and splicing the point feature and the global feature, and the ability to generate new feature tensors is limited.

[0133] And for the problems of PointNet, PointNet++ makes the following improvements and proposes a new neural network:

[0134] 1)PointNet++ uses the Farthest Point Sampling method and the query ball method, which pays more attention to the local features of the point cloud;

[0135] 2) Use a hierarchical feature learning framework to obtain different scale and level fusion point feature tensors using a step-by-step downsampling method;

[0136] 3) In the segmentation task, an encoder-decoder structure is constructed, which first down-samples and then up-samples, and uses residual connection to splice the fusion features of the corresponding layer.

[0137] Further, the step 2.1 is specifically to record the grasping pose as G, which is defined in detail as:

[0138] G=[R tω] T (5)

[0139] wherein, R∈R 3×3 represents the direction of the gripper; t∈R 3×1 , represents the position of the center when the gripper reaches the grasping pose; ω∈R, represents the width that should be opened when the gripper grasps the target;

[0140] It should be noted that R is a rotation matrix, because the determinant of the rotation matrix must be equal to 1, and the inverse of the matrix is the transpose of itself, it is difficult to directly input the rotation matrix R into the neural network for training, so R needs to be transformed and decoupled into viewpoint classification and in-plane rotation; At this time, the problem of generating a grasping pose for an object surface can be redefined as:

[0141] By predicting a feasible grasping point, based on this point, the vector that can approach this point, the distance of the gripper when clamping, and a gripper width that can clamp the target are predicted, and based on the predicted vector, the in-plane rotation in the direction of the vector is obtained; as Figure 6 shown;

[0142] The origin of the coordinate system is the center of the two-finger gripper, and the z-axis direction is the same as the approaching vector direction.

[0143] Further, the step 2.2 grasping prediction network includes an approaching network, an operation network and a tolerance network, the approaching network uses PointNet++ as the core network for viewpoint classification and feature extraction of point cloud;

[0144] N three-dimensional points are taken as input, a new point set with C-dimensional features can be realized by calling the encoder-decoder module of PointNet++, and the point cloud in the whole scene is down-sampled using the farthest point sampling method mentioned above, and then M points are output; the output of the proximity network is a vector of Mx(2+V) size, wherein 2 stores a binary class (True / False) representing whether it can be grabbed or not, and V represents the number of predefined proximity vectors;

[0145] The proximity network loss function is specifically defined as a binary label indicating whether it can be grabbed. For each predicted grabbing point, a binary label is assigned. For these predicted points, the ball query method in PointNet++ is used to construct a spherical space with a radius of r as the origin. The value of r is 0.005m in the present application. Search is performed in this spherical space. If at least one real point that can be grabbed exists, the point is regarded as a grabbable point, and its binary label is assigned as 1. Then, for each grabbable point, V virtual proximity vectors are sampled under the camera. The proximity vector of the ith grabbing point in the jth virtual view is denoted as v ij The objective function can be obtained as follows:

[0146]

[0147] Wherein, c i represents the prediction of whether the point can be grabbed. If it is positive, it is assigned as 1, and if it is negative, it is assigned as 0. s ij represents the prediction confidence score of point i in view j, is its corresponding real situation, represents the angle difference, which is limited to within 5° by the indicator function 1(), and for L cls The two-class softmax loss function, i.e. the binary cross-entropy loss function, is used, as shown in equation (7), for L reg To avoid the problems of loss convergence difficulty and out-of-control gradient value explosion, the present application uses the smooth L1 loss as shown in equation (8):

[0148]

[0149] Wherein, y is the sample label, is the probability that the model predicts the sample to be positive, and the sample is positive, taking the value of 0, and vice versa,

[0150]

[0151] Wherein, β is taken as 1;

[0152] Get the approaching vector v ij After that, further predict the in-plane rotation, approaching distance, gripper width and grasping confidence; all the grasping confidences are in the interval of (0, 1] and are divided into 10 categories, the closer to 0 represents the lower score, and vice versa; before inputting the feasible grasping candidates into the operation network, a unified representation is established for all the grasping candidates, which is converted into a (3+C) dimensional vector; if the approaching distance is divided into K intervals, then each distance interval can be represented as d K ; for a given d K , a cylinder is constructed with the approaching vector center, and a fixed number of sampling points are sampled in the cylinder; in order to facilitate neural network training, a new coordinate system is defined, taking the grasping point as the origin, v ij as the z-axis, and all the sampling points are converted into the new coordinate system, and the conversion matrix O ij is calculated as follows:

[0153]

[0154] wherein, and represent the 2nd element of v ij , and the same reasoning can be obtained.

[0155] Further, after the above transformation, the candidate grasping pose has a unified description, and then the in-plane rotation and the clamping width are predicted; for the in-plane rotation, the classification method is used to predict the effect, which is better than the regression. Therefore, the aligned point cloud is used as the input, and the classification score and the normalized residual error are predicted for each rotation, as well as the corresponding clamping width and the prediction confidence; since the invention is based on a two-finger gripper design, the gripper has axis symmetry in the geometric plane, so the invention only predicts the rotation angle in [0, 180°). For the design of the objective function, it is as follows:

[0156]

[0157] wherein R ij represents the rotation degree, S ij , W ij and d respectively represent the confidence score, the gripper width and the approaching distance. L d represents the loss of the dth distance.

[0158] Further, after the above steps, the end-to-end grasp prediction network can predict the grasp pose. However, in order to improve the robustness of grasp prediction, it is necessary to introduce the representation of grasp affinity fields. For a certain grasp point, a grasp pose can be rotated in any plane in space, and the rotated grasp pose is still feasible, which means that the feasible grasp pose of a single grasp point in space is almost infinite. In daily life, the grasp pose that humans tend to choose often has the characteristic of greater tolerance, that is, the grasp result according to the grasp pose has stronger robustness and anti-interference ability. Therefore, the present application needs to design a tolerance network, which needs to do is to predict the tolerance of the grasp pose like a human.

[0159] For this tolerance network, the following objective function is designed: for a given real grasp pose, search its neighborhood in the space to find the maximum distance at which the grasp is still robust and the grasp score is greater than 0.5, and set this behavior as the objective function.

[0160] Then the loss function can be written as:

[0161]

[0162] Where, T ij represents the maximum disturbance that the grasp pose can resist.

[0163] Further, the training of the network uses a K-neighbor algorithm, and the K-neighbor algorithm is specifically,

[0164] The K-neighbor algorithm (k-nearest neighbor classification, knn) is a commonly used classification and regression algorithm. The principle of knn can be simply summarized as follows: when predicting an unknown x, search around the x, compare the similarity of the k nearest points to it, and the category with the highest proportion is the classification of x. The K-neighbor algorithm is a famous lazy learning method, which does not need initial training or explicit learning process. In the training stage, only the training samples are saved, and the hyperparameters are not updated. The time cost is 0, and only when the test sample is received, the training sample is processed. For the K-neighbor algorithm, the selection of distance measurement and k value needs to be considered in actual use.

[0165] The distance between two instance points in the feature space is actually the similarity of the two points. The feature space is generally an n-dimensional real vector space. Assuming that there is an existing n-dimensional feature space H, x i and x j ∈R n , where xi and x j are the i-th and j-th points in space, respectively, and have The metric distance is usually a norm distance L p which can be calculated by equation (17):

[0166]

[0167] In the above formula, p≥1.

[0168] In practical applications, p=1 or 2 is usually selected, p=1 is Manhattan distance, and p=2 is Euclidean distance. The present application selects to use Euclidean distance for calculation.

[0169] The k of the K-neighbor algorithm is a hyperparameter of the algorithm, and the selection of the k value will have a great impact on the algorithm result. When the k value is small, the training sample used for prediction is reduced, and the approximation error is reduced. Only when the training sample is highly similar to the input test sample will it participate in the prediction process and affect the prediction result, but at this time the estimation error will often increase, and the generalization ability of the model will be poor. That is, the decrease of k value means the increase of the complexity of the model and the decrease of the robustness of the model, at this time the probability of overfitting phenomenon of the model is higher. Selecting a larger k value is equivalent to expanding the neighborhood, i.e. expanding the training sample used for prediction, and the training sample with lower similarity will also affect the prediction result, so a larger k value can reduce the estimation error, but will bring a larger approximation error. That is, the increase of k value reduces the complexity of the model and reduces the possibility of overfitting phenomenon.

[0170] When selecting the k value in practice, k is usually taken as an odd number to prevent two classes with the same number of classes from being unable to determine the object class.

[0171] Before starting the training, the multiple loss functions of the present application need to be combined, and the network realizes end-to-end update by minimizing the total loss function; as follows:

[0172] L = L A ({c i},{s ij}) + aL R (R ij ,S ij ,W ij ) + bL F (T ij ) (18)

[0173] Formula (18) is the total loss function of the network during training, wherein a and β are weights of different loss functions, and the purpose is to scale the orders of magnitude of the three losses to the same level, because generally, the convergence rates of different loss functions are different, and the sensitivity to different learning rates is also different, and unifying the individual losses to one order of magnitude can avoid the situation that a single loss is too large and the direction of model optimization is dominated by a single loss function. After debugging, it is determined that a is 1 and β is 0.2.

[0174] After obtaining the total target function of the network, the next step is to select an optimizer; the mechanism of the optimizer in deep learning is as follows: the optimizer mainly plays a role in the back propagation of training, guiding the parameters of the target function to update in the direction of gradient descent, so that the updated target function gradually approaches the global minimum; for the optimizer, there are usually the following input parameters: target function L, initial learning rate a, iteration number t and parameters to be optimized w; the basic steps of the optimizer to update the parameters are as follows:

[0175] 1) Calculate the gradient of the target function with respect to the current parameters:

[0176] grad t =▽L(w t ) (12)

[0177] 2) Calculate the first-order momentum and the second-order momentum according to the historical gradient:

[0178]

[0179] 3) Calculate the descent gradient at the current time:

[0180]

[0181] 4) Update the parameters according to the descent gradient:

[0182] w t+1 =w t -η t (20)

[0183] For the optimizer Adam used in the present application, the formula in the second step is different from the above; the Adam optimizer combines the SGDM and RMSProp optimizers, and the Adam optimizer introduces mechanisms such as iteration attenuation and adaptive learning rate, so formula (19) can be rewritten as:

[0184]

[0185] Then the parameter update formula of the Adam optimizer can be written as:

[0186]

[0187] Generally, m t and V t The m t and V t The estimated value is smaller than the true value at the initial stage, and the bias correction step is introduced to compensate for this deviation; the m t and V t are corrected to and As shown in formula (16):

[0188]

[0189] The Adam parameter update formula is Where epsilon is a small amount to prevent division by zero.

[0190] After determining the optimizer, the training of the grasping prediction network can be started. The training environment is shown in the figure: the system environment is Windows 11, the memory is 32G, and an RTX4060 8GB graphics card is used. Under the condition of setting the initial learning rate to 0.001, the results obtained by training are as follows:

[0191] From the figure, it can be seen that before the training stops, the curve of the loss function is basically stable, the trained result can be well fitted, and the parameters obtained by training can be used for subsequent experiments. Figure 11 The network evaluation is specifically,

[0192] The two abilities of the grasping prediction network are mainly explored: the prediction accuracy of the grasping prediction network for different objects under different scenes, that is, the generalization ability of the grasping prediction network is discussed; when the quality of the input point cloud is poor, such as uneven distribution of the point cloud and too sparse point cloud, the anti-interference ability of the grasping prediction network.

[0193] In order to evaluate the grasping prediction network, the present application designs a simulated grasping experiment, and the gripper selects a two-finger gripper, as shown in

[0194] The output accuracy of the grasping prediction network is represented by the success rate of grasping objects. Figure 12

[0195] ​The basic operation process is: according to the grasping point information given by the grasping prediction network, the highest score pose is directly selected for grasping, and after each grasping is executed, the gripper returns to the initial position, and the grasping success rate is used to evaluate the grasping pose generation ability of the grasping prediction network. Here, in order to discuss the grasping pose detection ability of the grasping prediction network for different parts of the satellite, two different scenes are designed, the friction coefficient is 0.3, which corresponds to three different parts of the satellite respectively, and single-view point cloud input is used in each scene.

[0196] As shown in Figure 13 , the first scene is for the docking ring part of the satellite, the initial position of the camera is directly above the docking ring, the three-dimensional coordinates are (0, 0, 1.65), and the three-dimensional coordinates of the satellite are (0, 0, 0) at this time.

[0197] In 50 grasping experiments, the successful times are 43 times, and the success rate is 86%.

[0198] As shown in Figure 14 , the second scene is for the sailboard part of the satellite, the initial three-dimensional coordinates of the camera are not changed, and the three-dimensional coordinates of the satellite are (0, 3.2, 0).

[0199] In 50 grasping experiments, the successful times are 46 times, and the success rate is 92%.

[0200] As shown in Figure 14 , the third scene is for the nozzle part of the satellite, the initial three-dimensional coordinates of the camera are not changed, and the three-dimensional coordinates of the satellite are (0, 1.2, 0.5).

[0201] Table 3-1 Grasping success rate of different parts of the satellite

[0202]

[0203] From the above table, it can be seen that the grasping pose detection accuracy of the grasping prediction network for the docking ring is lower than that for the sailboard. The reason is speculated as follows: the features of the sailboard part of the target satellite are relatively simple, and the network can better extract the features of the points of the sailboard part, while the features of the satellite docking ring are relatively complex, the information contained in the single-view point cloud is less, and there are occlusions in the single-view point cloud. The point cloud holes in the occluded part will affect the results generated by the grasping prediction network.

[0204] The grasping prediction network of the present application only takes point cloud as the only input, and is extremely dependent on the target point cloud. However, in the actual on-orbit capture task, it is inevitable to encounter the situation that the light condition under which the camera is located is not conducive, such as excessive exposure or satellite surface reflection, resulting in poor camera imaging quality, and thus poor quality of the generated point cloud, and problems such as point cloud holes and point cloud being too sparse. In the following, the present application will discuss the influence of the density of the point cloud on the accuracy of the network output.

[0205] Point clouds with different numbers of points are obtained as inputs of the network, as shown in Figure 16 The number of point clouds is gradually reduced from 8x10 4 to 5x10 3 These point clouds with different numbers of points are divided into 6 groups, and each group performs 50 grasps, and the success rate is as shown in Figure 17

[0206] It can be seen that the number of points in the point cloud has an acceptable influence on the accuracy of the network. When the number of points is kept at 1x10 4 and above, the accuracy does not fluctuate too much, and when the number of points is 5x10 3 , the accuracy decreases greatly.

[0207] Embodiment Two

[0208] The present embodiment obtains a spatial non-cooperative target capturable position autonomous intelligent decision system, which uses the spatial non-cooperative target capturable position autonomous intelligent decision method as described in Embodiment One, and the decision system comprises

[0209] a spatial target point cloud generation unit for generating a spatial target point cloud;

[0210] a grasping prediction network construction unit for constructing a grasping prediction network;

[0211] a network training unit for training the grasping prediction network constructed by the grasping prediction network construction unit;

[0212] using the grasping prediction network trained by the network training unit to predict the grasping of the spatial target point cloud generated by the spatial target point cloud generation unit.

[0213] Embodiment Three

[0214] ​The electronic device provided by the embodiment of the present application comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the memory is used for storing software programs and modules, and the processor executes various function applications and data processing by running the software programs and modules stored in the memory. The memory and the processor are connected through a bus. Specifically, the processor realizes any step in the embodiment one by running the computer program stored in the memory.

[0215] It should be understood that, in the embodiment of the present application, the processor can be a central processing unit (CPU), and the processor can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0216] The memory can comprise read-only memory, flash memory and random access memory, and provide instructions and data for the processor. Part or all of the memory can also comprise non-volatile random access memory.

[0217] As can be seen from the above, the electronic device provided by the embodiment of the present application can realize the autonomous intelligent decision-making method for the captureable position of the space non-cooperative target as described in the embodiment one by running the computer program, and obtain a new end-to-end network, called a grasping prediction network, for predicting the grasping of the space target point cloud.

[0218] It should be understood that the above-mentioned integrated modules / units, if implemented in the form of software functional units and sold or used as independent products, can be stored in a computer readable storage medium. Based on such understanding, all or part of the processes in the above-mentioned embodiments of the method can also be completed by a computer program instructing related hardware, and the above-mentioned computer program can be stored in a computer readable storage medium. When the processor executes the computer program, the steps of the above-mentioned various method embodiments can be implemented. The above-mentioned computer program includes computer program code, which can be in the form of source code, object code, executable files or some intermediate forms. The above-mentioned computer readable medium can include any entity or device capable of carrying the above-mentioned computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), electrical carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the above-mentioned computer readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction.

[0219] The above description of disclosed embodiments enables one of ordinary skill in the art to make or use the application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein can be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0220] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of functional units and modules is taken as an example, and in actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the above-mentioned device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiments can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or software functional unit. In addition, the specific names of each functional unit and module are only for easy distinction, and do not limit the protection scope of the application. The specific working process of the unit and module in the system can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0221] It should be noted that the method and details thereof provided by the above-mentioned embodiments can be combined with the apparatus and device provided by the embodiments, and mutual reference is not repeated.

[0222] Those skilled in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0223] In the embodiments provided by the present application, it should be understood that the disclosed apparatus / terminal device and method can be implemented in other ways. For example, the above-mentioned apparatus / device embodiments are merely illustrative, and the division of the above-mentioned modules or units is merely a logical function division, and an actual implementation can be divided in another way, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0224] The above-mentioned embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. An autonomous intelligent decision-making method for a space non-cooperative target capturable position, characterized in that, The generating method comprises the following steps, Step 1: generating a spatial target point cloud; Step 2: constructing a grasping prediction network; Step 3: training the grasping prediction network constructed in step 2; Step 4: predicting grasping for the spatial target point cloud generated in step 1 by using the grasping prediction network trained in step 3.

2. The method of claim 1, wherein, Step 1 specifically involves assuming the pixel coordinates P of the color image... rgb Represented as (u R ,v R ,z R ) T , where u R v R , z R Let p represent the x-coordinate, y-coordinate, and depth value in the camera coordinate system of the color image, respectively, and denote its projection onto the color image plane as p. rgb ; Pixel coordinates P of the depth image ir It can be represented as (u L ,v L ,z L ) T Let represent the x-coordinate, y-coordinate, and depth value in the camera coordinate system of the depth image, respectively, and let p be the projection of the image onto the image plane. ir From the pinhole imaging model of the camera, we can obtain: P rgb With P ir The spatial coordinates of the same point in different camera coordinate systems can be described by a rotation matrix R and a translation matrix T, that is: P rgb = RP ir + T (2) Wherein the rotation matrix and the translation matrix are obtained from the external parameter of camera calibration, and the color image and the depth image are registered after coordinate transformation, that is, the RGB-D image can be obtained; According to the intrinsic matrix of the camera and the pixel point coordinates (u, v) in the pixel coordinate system provided by the color image, the coordinates (x, y) of the pixel point in the camera coordinate system can be obtained. The depth image can directly provide the depth z of the pixel point in the camera coordinate system, and thus the three-dimensional coordinates P=(x, y, z) in the camera coordinate system can be obtained. The relationship between the point in the camera coordinate system P and the point in the pixel coordinate system P uv may be expressed as: Where H represents the camera's intrinsic parameter matrix, f x f is the focal length of the camera along the x-axis. y Let c be the focal length of the camera along the y-axis. x and c y These are the x and y coordinates of the base point, respectively, and are the centers along the x and y axes of the image plane. z = d is the depth value measured by the camera. According to the homogeneous transformation matrix T from the world coordinate system to the pixel coordinate system, the following can be obtained: Wherein R is the rotation matrix of the world coordinate system to the pixel coordinate system, and t is the corresponding translation; thus the point cloud coordinates in the world coordinate system can be obtained.

3. The method of claim 1, wherein, The step 2 specifically comprises the following steps, Step 2.1: defining a two-finger gripper robot grasping posture and its coordinate system; Step 2.2: constructing a grasping prediction network based on the grasping posture and the coordinate system in step 2.

1.

4. The method of claim 3, wherein, The step 2.1 is specifically that the grasping posture is recorded as G, and is defined in detail as follows: G = [R t ω] T (5) where R ∈ R 3×3 represents the direction of the gripper; t ∈ R 3×1 represents the position of the center when the gripper reaches the grasping pose; ω ∈ R, represents the width that the gripper should open when it grasps the target; A feasible grasping point is obtained by prediction, based on which a vector capable of approaching the point, a distance of the gripper when clamping and a gripper width capable of clamping the target are predicted, and a plane in the direction of the vector is obtained based on the predicted vector; The origin of the coordinate system is the center of the two-finger gripper, and the z-axis direction is the same as the approaching vector direction.

5. The method of claim 3, wherein, The grasping prediction network in step 2.2 comprises an approaching network, an operation network and a tolerance network, the approaching network uses PointNet++ as a core network, and is used for viewpoint classification and extraction of point cloud features; N three-dimensional points are taken as inputs, an encoder-decoder module of PointNet++ is called, and a new point set with C-dimensional features can be realized; And the farthest point sampling method mentioned in the foregoing is used to down-sample the point cloud in the whole scene, and then M points are output; the output of the approaching network is a vector with a size of Mx(2+V), wherein 2 stores a binary category representing graspable or ungraspable, and V represents the number of predefined approaching vectors; The proximity network loss function is specifically defined as follows: a binary label is defined to represent whether it is graspable, and a binary label is assigned to each predicted grasp point. For these predicted points, the method of ball query in PointNet++ is used to construct a spherical space with a radius of r with the point as the origin. If there is at least one real point that is graspable in the spherical space, the point is regarded as a graspable point, and its binary label is assigned as 1. Then, for each graspable point, V virtual proximity vectors are sampled under the camera. The proximity vector of the jth virtual view of the ith grasp point is denoted as v ij , and the objective function is as follows: where c i represents the prediction of whether a point is graspable, and is assigned a value of 1 if positive, and 0 if negative represents the prediction of whether a point is graspable, and is assigned a value of 1 if positive, and 0 if negative ij represents the prediction confidence score of point i for viewpoint j, is its corresponding ground truth, represents the angular difference, and is limited to within 5° of the search range by the indicator function 1(), for L cls is trained using a binary softmax loss function, as shown in equation (7), for L reg is trained using a smooth L1 loss, as shown in equation (8): where y is the sample label, is the probability that the model predicts the sample to be positive, and the sample is positive, taking the value 0, otherwise taking the negative value, Wherein β is taken as 1; Get the proximity vector v ij After that, all the grasp confidences are in the interval of (0, 1] and are divided into 10 classes, the closer to 0 represents the lower score, and vice versa; before inputting the feasible grasp candidates into the operation network, they are converted into (3+C)-dimensional vectors; if the proximity distance is divided into K intervals, each distance is represented as d K ; for a given d K , a cylinder is constructed with the proximity vector center, and a fixed number of sampling points are sampled in the cylinder; a new coordinate system is defined with the grasp point as the origin, v ij as the z-axis, and all the sampling points are converted into the new coordinate system, the conversion matrix O ij is calculated as follows: wherein and denotes v ij the 2nd element of by analogy.

6. The method of claim 5, wherein, For in-plane rotation, the prediction effect of the classification method is better than that of the regression; the aligned point cloud is taken as input, classification scores and normalized residuals are predicted for each rotation, and corresponding clamping width and prediction confidence are predicted; since the clamping device is axisymmetric in the geometric plane, only the rotation angle in [0, 180°) is predicted; the objective function is as follows: where R ij represents the degree of rotation, S ij , W ij and d represent the confidence score, gripper width and proximity distance, respectively. L d represents the loss for the dth distance.

7. The method of claim 6, wherein, The tolerance network is specifically that for a given real grasping posture, the neighborhood in the space is searched to find the maximum distance with which the grasping is still robust and the grasping score is greater than 0.5, and the grasping behavior is set as the objective function; The loss function can be written as: where T ij represents the maximum disturbance that the grasping pose can resist.

8. The method of claim 1, wherein, The step 3 comprises data set preprocessing and network training. The data set preprocessing specifically comprises: first, obtaining a depth map and a color map of different perspectives of a space target based on Blender to obtain point cloud information, sampling to obtain a grasping point from the point cloud, then sampling and evaluating a grasping pose and a gripper grasping depth; finally, projecting the grasping pose onto a scene by using a 6D pose of each object; The 3D model and data used are placed in the GraspNet-1Billion data set, and the data set is arranged into a structure that can be imported; Finally, 200 scenes of the data set are divided, and the first 100 scenes are used as a training set, and the last 100 scenes are used as an evaluation set.

9. The method of claim 8, wherein, The network training specifically comprises: obtaining a total objective function of the network, and then selecting an optimizer; input parameters of the optimizer are an objective function L, an initial learning rate a, an iteration number t and parameters to be optimized w; basic steps of updating parameters of the optimizer are as follows: 1) calculating a gradient of the objective function with respect to current parameters; grad t = ∇L(w t ) (12) 2) calculating a first order momentum and a second order momentum according to historical gradients; 3) calculating a descending gradient at a current time; 4) updating parameters of the Adam optimizer according to the descending gradient: m t and V t The m t and V t Start from 0; m t and V t Corrected to and As shown in equation (16): The Adam parameter update formula is where ε is a small quantity to prevent the denominator from being zero.

10. A computer device, comprising: A computer program product, comprising a memory and a processor, and a computer program stored on the memory and capable of running on the processor, wherein the processor executes the computer program to implement the method according to any one of claims 1-9. A computer program product, comprising a memory and a processor, and a computer program stored on the memory and capable of running on the processor, wherein the processor executes the computer program to implement the method according to any one of claims 1-9.

Images