A method and system for industrial robot target recognition based on deep learning

Abstract

The present invention discloses an industrial robot target recognition method and system based on deep learning, comprising: firstly, using a depth camera to collect a depth image of a target object, and converting it into three-dimensional point cloud data of the target object, and marking the type of the target object, and constructing a target object database D; then using the depth camera to collect depth images of multiple known multi-object stacking scenes, and converting them into scene three-dimensional point cloud data, and marking the position and posture of the target object to be identified in the scene, and constructing a scene database Dc; constructing a scene semantic segmentation model, a target type discrimination model and a target posture recognition model; using the previously constructed D and Dc, sequentially training three deep learning models; finally, using the depth camera to collect the scene three-dimensional point cloud data of the multi-object stacking scene to be identified, and using the trained model to identify the scene to be identified.

Description

Technical Field

[0001] The present invention relates to the technical field of robot target recognition, and in particular to an industrial robot target recognition method and system based on deep learning. Background Art

[0002] In modern manufacturing and industrial applications, industrial robots play an important role in automated and intelligent production processes. Among them, the robot's target recognition and perception capabilities are crucial to safety, accuracy and efficiency. Misidentification of workpieces and workers' helmets can lead to subsequent operations that damage the workpiece at best and injure workers at worst, causing serious consequences. The rapid development of deep learning technology has greatly promoted the development of the field of industrial robots, providing robot systems with more advanced target recognition capabilities and more flexible target recognition methods.

[0003] However, most existing technologies are based on target recognition work based on images and two-dimensional traditional convolution operations, lacking the ability to perceive three-dimensional coordinates in the real world. Existing three-dimensional object detection methods are mainly based on three-dimensional images and three-dimensional convolution technology, but this technology has too high requirements on GPUs. Taking a 256-pixel three-dimensional image and U-Net as an example, at least 20G of video memory is required, and the training time for a single image model is up to 2 hours; there are also a few works based on point clouds, which are mainly based on improvements in point pair feature technology (PPF) and closest point iteration technology (ICP), but these two technologies have high requirements for data integrity and are easily affected by factors such as occlusion, workpiece texture, and lighting. Summary of the invention

[0004] In view of this, the present invention provides an industrial robot target recognition method and system based on deep learning, the purpose of which is to 1) develop an industrial robot target recognition system that can accurately identify target objects in these complex stacking scenarios even if only a portion of the target object is exposed. This function is very critical to improving production efficiency and reducing human intervention. For example, industrial robots can accurately identify and extract required parts from piles of goods without having to clear or rearrange materials in advance; 2) ensure that industrial robots have high-precision target recognition capabilities. Specifically, the system needs to identify the exact position and posture of the target object so that the robot can perform high-precision 3) Develop lightweight and efficient deep learning models specifically for point cloud data processing. These models can not only run on relatively low hardware configurations, but also process point cloud data efficiently to reduce the cost and energy consumption of robot systems. By achieving this goal, more industrial robots can benefit from advanced target recognition technology without expensive hardware upgrades, which is especially beneficial for small manufacturing companies or resource-constrained environments.

[0005] To achieve the above object, the present invention provides an industrial robot target recognition method based on deep learning, comprising the following steps:

[0006] Step S1: Use a depth camera to collect a depth image of the target object to be identified, and convert it into three-dimensional point cloud data of the object, mark the type of the target object, and build a target object database D;

[0007] Step S2: Use a depth camera to collect multiple known multi-object stacked scene depth images, and convert them into scene three-dimensional point cloud data, mark the position and posture of the target object to be identified in the scene, and build a scene database Dc;

[0008] Step S3: constructing a scene semantic segmentation model based on the scene 3D point cloud data, constructing a target type discrimination model based on the object 3D point cloud data, and constructing a target posture recognition model based on the scene 3D point cloud data;

[0009] Step S4: Use the target object database D and scene database Dc constructed by S1 and S2 to train the scene semantic segmentation model, target category discrimination model and target posture recognition model constructed by S3 in sequence;

[0010] Step S5: Use a depth camera to collect scene 3D point cloud data of a multi-object stacking scene to be identified, and use the three models trained in S4 to perform recognition tests on the scene that requires target recognition. If the test requirements are met, save the model data; if the test requirements are not met, return to S4 for retraining;

[0011] The depth camera is used to collect depth images of target objects or scenes, and has the ability to transmit the images to a data processing server in real time.

[0012] As a further improvement method of the present invention:

[0013] Optionally, the process of converting the depth image of the target object to be identified in step S1 into the three-dimensional point cloud data of the object is to convert the two-dimensional coordinates and depth value of each pixel point of the depth image into the three-dimensional coordinates and color value of the point cloud data. The present invention uses the three-dimensional point cloud data in the world coordinate system; this process requires the intrinsic parameters and extrinsic parameters of the depth camera; the intrinsic parameters of the camera are some fixed parameters, which describe the internal structure and imaging characteristics of the camera, such as focal length, principal point, distortion, etc.; the intrinsic parameters of the depth camera are represented by a 3x3 matrix N, and its form is as follows:

[0014]

[0015] Among them, j x is the horizontal focal length of the camera, j y is the vertical focal length of the camera, z x is the horizontal coordinate of the camera's principal point, z y is the ordinate of the principal point of the camera; the intrinsic parameter matrix N of the depth camera converts the image coordinates of a two-dimensional point on a depth image into the coordinates of a three-dimensional point cloud data point of an object in the camera coordinate system. The formula is as follows:

[0016]

[0017] Among them, sx, sy, sz are the specific coordinate information of the three-dimensional point cloud in the depth camera coordinate system, eu, ev are the coordinate information of the two-dimensional depth image, ez is the depth information of the two-dimensional depth image at the position of eu, ev, N -1 is the inverse matrix of the camera's intrinsic parameter matrix N; the extrinsic parameters of the depth camera are some variable parameters, which describe the position and posture of the camera in the world coordinate system, such as translation and rotation. The extrinsic parameters are represented by a 4x4 matrix W:

[0018]

[0019] Among them, X is a 3x3 rotation matrix, q is a 3x1 translation vector; the camera's external parameter matrix W converts the three-dimensional point coordinates in a camera coordinate system to the three-dimensional point coordinates in a world coordinate system. The formula is as follows:

[0020]

[0021] Among them, X -1 is the inverse matrix of the rotation matrix X, gx, gy, and gz are the specific coordinate information of the three-dimensional point cloud in the world coordinate system.

[0022] In the step S1, the object types are marked and the target object database is constructed by establishing a correspondence between the three-dimensional point cloud data P of the object in the world coordinate system and the actual category C of the object, and obtaining the target object database D. The target database D contains n groups of data, where n is the total number of target object types;

[0023] D={[P1,C1],[P2,C2],...,[P n ,C n ]}

[0024] Among them, all subscripts refer to the order, P1…P n is the 3D point cloud data of different objects, C1…C n is the actual category C of the object point cloud data.

[0025] Optionally, the process of converting the scene depth map into the scene three-dimensional point cloud data in step S2 is the same as that in step S1. In step S2, the position and posture of the target object to be identified in the scene are marked. The construction of the scene database is to establish the correspondence between the scene three-dimensional point cloud data Po in the world coordinate system, the scene semantic label Lb, and the type C, position pos, and posture ges of all target objects contained in the scene, and obtain the scene database Dc, Dc contains m groups of data, and m is the number of scenes collected;

[0026]

[0027] Among them, Po1…Po m Three-dimensional point cloud data Po, Lb1…Lb for different scenes m are the corresponding scene semantic labels Lb, C m1 …C mk Po for scene 3D point cloud data m The actual category C of each object in pos m1 …pos mk Po for scene 3D point cloud data m The position of each object in pos,ges m1 …ges mkPo for scene 3D point cloud data m The posture of each object in ges.

[0028] Optionally, the step S3 constructs a scene semantic segmentation model based on point cloud data, and the specific structure is:

[0029] The scene semantic segmentation model includes a point cloud feature extraction module NetA, a multi-scale grouping module NetB, a feature fusion module NetC and a segmentation output head module NetOut;

[0030] The point cloud feature extraction module NetA includes multiple point cloud feature extraction submodules, each of which includes a point cloud sampling layer and two convolutional layers connected in series; the convolutional layer adopts an MLP structure, the MLP dimensions of the first convolutional layer are 64, 64, 128, and the MLP dimensions of the second convolutional layer are 128, 128, 256; the NetA input is the scene point cloud data Po of Num x 3, and the output is a high-dimensional feature vector set GT1 of Ke x 256, where Num is the number of points in the point cloud, and Ke is the number of key points after sampling;

[0031] GT1=NetA(Po)

[0032] The multi-scale grouping module NetB includes multiple parallel sub-modules, each sub-module corresponds to a neighborhood radius of a scale, and a scale is generally regarded as a quantity that is initialized at any time; each sub-module includes a sampling layer, a spherical query layer, and two convolutional layers in series; the convolutional layer adopts an MLP structure, the MLP dimensions of the first convolutional layer are 128, 128, 256, and the MLP dimensions of the second convolutional layer are 256, 256, 512; the NetB input is a feature vector set GT1 of Ke x 256, and the output is a multi-scale feature vector set GT2 of Ke x 512;

[0033] GT2=NetB(GT1)

[0034] The feature fusion module NetC is a point-by-point feature fusion network, comprising multiple convolutional layers connected in series, and the MLP dimensions of each convolutional layer are 256, 256, and 512; the input of NetC is a multi-scale feature vector set GT2 of Ke x 512, and the output is a fused feature vector set GT3 of Ke x 512;

[0035] GT3=NetC(GT2)

[0036] The segmentation output head module NetOut contains multiple convolutional layers, the MLP dimensions of each convolution layer are 512, 256, 128, and the last layer is a 1x1 convolutional layer, which is used to map the feature vector to the category of each point; the NetOut input is a fusion feature vector set of Ke x 512, and the output is a classification result R_seg of Num x Cas, where Num is the number of points in the point cloud, and Cas is the number of scene semantic segmentation categories. The specific calculation method is:

[0037] R_seg=NetOut(GT3)

[0038] Cas=n+1

[0039] Among them, n is the total number of target objects.

[0040] The step S3 constructs a target type discrimination model based on point cloud data, and the specific structure is:

[0041] The target type discrimination model includes a feature extraction module NetE, a low-dimensional feature fusion module NetF, a high-dimensional feature fusion module NetG and a classification output head module NetH; the number of times the target discrimination model runs is equal to the number of target objects Fn recognized by the scene semantic segmentation model;

[0042] The feature extraction module NetE includes a six-layer network in series, which is used to preliminarily extract the features of point cloud data; the 1st, 3rd and 5th layers are convolution layers, the number of convolution kernels are 8, 16 and 32 respectively, the convolution kernel size is 3, the step size is 1, the padding mode is "same", and the activation function uses ReLU; the 2nd, 4th and 6th layers are pooling layers, using maximum pooling, the pooling kernel size is 2x2, the step size is 2, and the padding mode is "same"; the input of NetE is a single target point cloud R_seg_part in the semantic segmentation result output by the scene semantic segmentation model, which is Ns×3 point cloud data, and the output is a primary feature vector DT1 of Ns×32, where Ns is the number of points in R_seg_part;

[0043] R_seg_part∈R_seg

[0044] DT1=NetE(R_seg_part)

[0045] The low-dimensional feature fusion module NetF further fuses and processes the point cloud features; NetF includes two parallel feature fusion subnetworks NetF1 and NetF2; each subnetwork contains 10 layers of networks with the following structure: the 1st, 3rd, 5th, 7th, and 9th layers are convolution layers, the number of convolution kernels are 32, 64, 128, 256, and 512 respectively, the convolution kernel size is 1, the step size is 1, the padding mode is "same", and the activation function uses ReLU; the 2nd, 4th, 6th, 8th, and 10th layers are pooling layers, using maximum pooling, the pooling kernel size is 2, the step size is 2, and the padding mode is "valid"; the input of NetE is the output DT1 of NetF, and the output is the N×512 secondary feature vector DT2;

[0046] DT2=NetF(DT1)=NetF1(DT1)⊕NetF2(DT2)

[0047] Among them, ⊕ is the sum of the eigenvectors in element order;

[0048] The high-dimensional feature fusion module NetG improves the spatial resolution of features by upsampling and fusion; NetG includes three deconvolution layers, the number of convolution kernels are 256, 128, and 64 respectively, the convolution kernel size is 3, the step size is 2, the padding mode is "same", and the activation function uses ReLU; the input of NetG is the output DT2 of NetF, and the output is the feature secondary feature vector DT3 of N×64;

[0049] DT3=NetG(DT2)

[0050] The classification output head module NetH is used to generate the final classification result; NetH contains two layers of networks, the first layer is a convolution layer, the number of convolution kernels is 64, the convolution kernel size is 1, the step size is 1, the padding mode is "same", and the activation function uses ReLU; the second layer is a convolution layer, the number of convolution kernels is the total number of target object types n, the convolution kernel size is 1, the step size is 1, the padding mode is "same", and the SoftMax activation function is used; the input of NetH is the output DT3 of NetG, and the output is R_cls, which is the classification probability of a single target point cloud R_seg_part in each category;

[0051] R_cls=NetH(DT3).

[0052] The step S3 constructs a target posture recognition model based on point cloud data, and the specific structure is:

[0053] The target posture recognition model includes a feature extraction module NetI and a posture recognition output head module NetJ; the number of times the target posture recognition model runs is equal to the number of target objects Fn recognized by the scene semantic segmentation model;

[0054] The feature extraction module NetI includes a 6-layer network, which is specially designed to extract key features from point cloud data; the 1st, 3rd and 5th layers are convolution layers, the number of convolution kernels are 8, 16 and 32 respectively, the convolution kernel size is 3, the step size is 1, the padding mode is "same", and the activation function uses ReLU; the 2nd, 4th and 6th layers are pooling layers, which use the maximum pooling mode, the pooling kernel size is 2, the step size is 2, and the padding mode is "same"; the input of NetI is Ns×3 point cloud data, and the output is Ns×32 posture feature vector ZT1, where Ns is the number of points in R_seg_part;

[0055] ZT1=NetI(R_seg_part)

[0056] The posture recognition output head module NetJ is used to generate the final posture recognition result; NetJ contains two layers of networks, the first layer is a convolution layer, the number of convolution kernels is 64, the convolution kernel size is 1, the step size is 1, the filling method is "same", and the activation function uses ReLU; the second layer is a fully connected layer, and the output is a 6-dimensional vector loc_ges, the first three dimensions represent the position, and the last three dimensions represent the posture;

[0057] loc_ges=NetJ(ZT1).

[0058] Optionally, the scene semantic segmentation model, the target type discrimination model and the target posture recognition model of step S4 are trained in sequence, specifically including the following three steps:

[0059] Step S401, training the semantic segmentation model, using the scene database Dc constructed in step S2 as training data, training the semantic segmentation model, and using the cross entropy loss as the loss function of the training process. The specific calculation formula is:

[0060]

[0061] Among them, Num is the number of points in the point cloud, Cas is the number of categories, r ij is the one-hot encoding of the semantic label Lb in the scene database flattened into one row, indicating that the i-th point belongs to the j-th category, t ij is the prediction result output by the deep learning model. The value represents the probability that the i-th point belongs to the j-th category. The meanings of i and j are the same as those of r. ij The meaning is the same; the optimizer of the training process uses the Adam optimizer, and the learning rate α1 ranges from:

[0062] α1∈[0.0001,0.01]

[0063] Step S402, training a target type discrimination model, includes:

[0064] Step B1, data enhancement of the target object database D, including random cropping, rotation, and scaling, where the random cropping, rotation, and scaling operations are all performed on the point cloud data P, and finally an enhanced target object database D2 with n2 point cloud data P is obtained, where n2>>n, and the value range of n2 is [100, 1000]. 90% of the obtained target object database D2 is used as training data KD_train, and 10% is used as test data KD_test.

[0065] KD=Data_aug(D2)

[0066] =Data_aug({P,C})

[0067] ={Data_aug(P),Data_aug(C)}

[0068] KD_train,KD_test=random_split(KD,0.9,0.1)

[0069] Among them, Data_aug() represents random cropping, flipping, rotation, and scaling of the point cloud, and random_split() represents the proportional division of the data set, and the proportion is represented by the last two elements;

[0070] Step B2: Use KD_train to train the target category discrimination model. The loss function of the training process uses the cross entropy loss. The specific calculation formula is:

[0071]

[0072] Among them, n2 is the number of data in the target object database D2, n is the number of target object types, r qw The qth element in the target object database D2 belongs to the wth category, t qw is the prediction result output by the target type discrimination model, t qw ∈[0,1], the value represents the probability of the existence or non-existence of scratches, and the meanings of q and w are the same as those of r qw The meaning is the same; the optimizer of the training process uses the Adam optimizer, and the learning rate α2 range is:

[0073] α2∈[0.0001,0.01]

[0074] Step S403, training the target posture recognition model, using the scene database Dc constructed in step S2 as training data, to train the target posture recognition model. The training process uses the L1 loss function for the position information training and the L2 loss function for the posture information training. The specific calculation formula is:

[0075] L1=|loc_ges[:3]-pos|

[0076]

[0077] Among them, loc_ges is the 6-dimensional vector output by the target posture recognition model, pos is the position information corresponding to R_seg_part in the scene database, and ges is the posture information corresponding to R_seg_part in the scene database; the optimizer of the training process uses the SGD optimizer, and the learning rate α3 range is:

[0078] α3∈[0.0001,0.01].

[0079] Optionally, in step S5, the three models trained in S4 are used to perform recognition tests on scenes requiring target recognition. If the test requirements are met, the model data is saved; if the test requirements are not met, the model is returned to S4 for retraining. The mIOU evaluation index is used for the scene semantic segmentation model, the global accuracy acc evaluation index is used for the target type discrimination model, and the posture error err evaluation index is used for the posture recognition model:

[0080] err=L1+L2

[0081] Among them, L1 is the loss function for position information, and L2 is the loss function for posture information.

[0082] The trained deep learning model can be used for target recognition and estimation of position and posture information.

[0083] The present invention also provides a robot target recognition system based on deep learning, comprising:

[0084] Depth camera: used to obtain depth images of target objects and scenes, and internal and external parameters must be known in advance;

[0085] Industrial computer: used to obtain depth images and convert them into three-dimensional point cloud information, train three deep learning models provided by the robot target recognition method based on the deep learning method and store model data, test the three deep learning models, and drive the robot to perform operations;

[0086] Robot: used to carry depth cameras and industrial computers.

[0087] To realize the above-mentioned industrial robot target recognition method based on deep learning.

[0088] The beneficial effects brought about by the technical solution provided by the present invention include at least:

[0089] Optimization of data processing: By using the depth images collected by the depth camera and converting them into three-dimensional point cloud data, combined with scene semantic segmentation, target type discrimination and posture recognition, this system has high efficiency and accuracy in data processing and model training, optimizing the entire recognition process.

[0090] Improve recognition accuracy and efficiency: By combining deep learning technology and 3D point cloud data processing, the system can accurately identify target objects in complex stacking scenes. Even if only a portion of the target object is exposed, the system can accurately identify it. This high-precision recognition function is critical to improving production efficiency and reducing manual intervention.

[0091] Enhanced precision and safety of operations: The system can identify the exact position and posture of the target object, which is critical for performing precision tasks such as assembly, welding, cutting and quality inspection. Through accurate recognition and operation, the risk of workpiece damage and worker injury caused by misidentification can be significantly reduced.

[0092] Reduced cost and energy consumption: The lightweight deep learning model developed in this invention is designed to reduce the reliance on high-performance hardware. This allows the system to not only run on lower hardware configurations, but also process point cloud data efficiently, thereby reducing the overall cost and energy consumption of the robotic system. This system is particularly suitable for small manufacturing companies or resource-constrained environments, enabling more industrial robots to benefit from advanced object recognition technology without expensive hardware upgrades. BRIEF DESCRIPTION OF THE DRAWINGS

[0093] Figure 1 The present invention is a flowchart of a method and system for industrial robot target recognition based on deep learning according to an embodiment of the present invention. DETAILED DESCRIPTION

[0094] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0095] Embodiment 1:

[0096] In order to achieve the above objectives, the present invention provides an industrial robot target recognition method and system based on deep learning, such as Figure 1 The following steps are shown:

[0097] Step S1: Use a depth camera to collect a depth image of the target object to be identified, and convert it into three-dimensional point cloud data of the object, mark the type of the target object, and build a target object database D;

[0098] The process of converting the depth image of the target object to be identified in step S1 into the three-dimensional point cloud data of the object is to convert the two-dimensional coordinates and depth value of each pixel point of the depth image into the three-dimensional coordinates and color value of the point cloud data. The present invention uses the three-dimensional point cloud data in the world coordinate system; this process requires the intrinsic parameters and extrinsic parameters of the depth camera; the intrinsic parameters of the camera are some fixed parameters, which describe the internal structure and imaging characteristics of the camera, such as focal length, principal point, distortion, etc.; the intrinsic parameters of the depth camera are represented by a 3x3 matrix N, and its form is as follows:

[0099]

[0100] Among them, j x is the horizontal focal length of the camera, j y is the vertical focal length of the camera, z x is the horizontal coordinate of the camera's principal point, z y is the ordinate of the principal point of the camera; the intrinsic parameter matrix N of the depth camera converts the image coordinates of a two-dimensional point on a depth image into the coordinates of a three-dimensional point cloud data point of an object in the camera coordinate system. The formula is as follows:

[0101]

[0102] Among them, sx, sy, sz are the specific coordinate information of the three-dimensional point cloud in the depth camera coordinate system, eu, ev are the coordinate information of the two-dimensional depth image, ez is the depth information of the two-dimensional depth image at the position of eu, ev, N -1 is the inverse matrix of the camera's intrinsic parameter matrix N; the depth camera's extrinsic parameters are represented by a 4x4 matrix W:

[0103]

[0104] Among them, X is a 3x3 rotation matrix, q is a 3x1 translation vector; the camera's external parameter matrix W converts the three-dimensional point coordinates in a camera coordinate system to the three-dimensional point coordinates in a world coordinate system. The formula is as follows:

[0105]

[0106] Among them, X -1 is the inverse matrix of the rotation matrix X, gx, gy, and gz are the specific coordinate information of the three-dimensional point cloud in the world coordinate system.

[0107] In the step S1, the object types are marked and the target object database is constructed by establishing a correspondence between the three-dimensional point cloud data P of the object in the world coordinate system and the actual category C of the object, and obtaining the target object database D. The target database D contains n groups of data, where n is the total number of target object types;

[0108] D={[P1,C1],[P2,C2],...,[P n ,C n ]}

[0109] Among them, all subscripts refer to the order, P1…P n is the 3D point cloud data of different objects, C1…C n is the actual category C of the object point cloud data.

[0110] Step S2: Use a depth camera to collect multiple known multi-object stacked scene depth images, and convert them into scene three-dimensional point cloud data, mark the position and posture of the target object to be identified in the scene, and build a scene database Dc;

[0111] The process of converting the scene depth map into the scene three-dimensional point cloud data in step S2 is the same as that in step S1. In step S2, the position and posture of the target object to be identified in the scene are marked. The construction of the scene database is to establish the corresponding relationship between the scene three-dimensional point cloud data Po in the world coordinate system, the scene semantic label Lb, and the type C, position pos, and posture ges of all target objects contained in the scene, and obtain the scene database Dc, Dc contains m groups of data, and m is the number of scenes collected;

[0112]

[0113] Among them, Po1…Po m Three-dimensional point cloud data Po, Lb1…Lb for different scenes m are the corresponding scene semantic labels Lb, C m1 …C mk Po for scene 3D point cloud data m The actual category C of each object in pos m1 …pos mk Po for scene 3D point cloud data m The position of each object in pos,ges m1 …ges mk Po for scene 3D point cloud data m The posture of each object in ges.

[0114] Step S3: constructing a scene semantic segmentation model based on the scene 3D point cloud data, constructing a target type discrimination model based on the object 3D point cloud data, and constructing a target posture recognition model based on the scene 3D point cloud data;

[0115] The step S3 constructs a scene semantic segmentation model based on point cloud data, and the specific structure is:

[0116] The scene semantic segmentation model includes a point cloud feature extraction module NetA, a multi-scale grouping module NetB, a feature fusion module NetC and a segmentation output head module NetOut;

[0117] The point cloud feature extraction module NetA includes multiple point cloud feature extraction submodules, each of which includes a point cloud sampling layer and two convolutional layers connected in series; the convolutional layer adopts an MLP structure, the MLP dimensions of the first convolutional layer are 64, 64, 128, and the MLP dimensions of the second convolutional layer are 128, 128, 256; the NetA input is the scene point cloud data Po of Num x 3, and the output is a high-dimensional feature vector set GT1 of Ke x 256, where Num is the number of points in the point cloud, and Ke is the number of key points after sampling;

[0118] GT1=NetA(Po)

[0119] The multi-scale grouping module NetB includes multiple parallel sub-modules, each sub-module corresponds to a neighborhood radius of a scale, and a scale can be regarded as a quantity that can be initialized at any time in this embodiment; each sub-module includes a sampling layer, a spherical query layer, and two convolutional layers in series; the convolutional layer adopts an MLP structure, the MLP dimensions of the first convolutional layer are 128, 128, 256, and the MLP dimensions of the second convolutional layer are 256, 256, 512; the NetB input is a feature vector set GT1 of Ke x 256, and the output is a multi-scale feature vector set GT2 of Ke x 512;

[0120] GT2=NetB(GT1)

[0121] The feature fusion module NetC is a point-by-point feature fusion network, comprising multiple convolutional layers connected in series, and the MLP dimensions of each convolutional layer are 256, 256, and 512; the input of NetC is a multi-scale feature vector set GT2 of Ke x 512, and the output is a fused feature vector set GT3 of Ke x 512;

[0122] GT3=NetC(GT2)

[0123] The segmentation output head module NetOut contains multiple convolutional layers, the MLP dimensions of each convolution layer are 512, 256, 128, and the last layer is a 1x1 convolutional layer, which is used to map the feature vector to the category of each point; the NetOut input is a fusion feature vector set of Ke x 512, and the output is a classification result R_seg of Num x Cas, where Num is the number of points in the point cloud, and Cas is the number of scene semantic segmentation categories. The specific calculation method is:

[0124] R_seg=NetOut(GT3)

[0125] Cas=n+1

[0126] Among them, n is the total number of target objects.

[0127] The step S3 constructs a target type discrimination model based on point cloud data, and the specific structure is:

[0128] The target type discrimination model includes a feature extraction module NetE, a low-dimensional feature fusion module NetF, a high-dimensional feature fusion module NetG and a classification output head module NetH; the number of times the target discrimination model runs is equal to the number of target objects Fn recognized by the scene semantic segmentation model;

[0129] The feature extraction module NetE includes a six-layer network in series, which is used to preliminarily extract the features of point cloud data; the 1st, 3rd and 5th layers are convolution layers, the number of convolution kernels are 8, 16 and 32 respectively, the convolution kernel size is 3, the step size is 1, the padding mode is "same", and the activation function uses ReLU; the 2nd, 4th and 6th layers are pooling layers, using maximum pooling, the pooling kernel size is 2x2, the step size is 2, and the padding mode is "same"; the input of NetE is a single target point cloud R_seg_part in the semantic segmentation result output by the scene semantic segmentation model, which is Ns×3 point cloud data, and the output is a primary feature vector DT1 of Ns×32, where Ns is the number of points in R_seg_part;

[0130] R_seg_part∈R_seg

[0131] DT1=NetE(R_seg_part)

[0132] The low-dimensional feature fusion module NetF further fuses and processes the point cloud features; NetF includes two parallel feature fusion subnetworks NetF1 and NetF2; each subnetwork contains 10 layers of networks with the following structure: the 1st, 3rd, 5th, 7th, and 9th layers are convolution layers, the number of convolution kernels are 32, 64, 128, 256, and 512 respectively, the convolution kernel size is 1, the step size is 1, the padding mode is "same", and the activation function uses ReLU; the 2nd, 4th, 6th, 8th, and 10th layers are pooling layers, using maximum pooling, the pooling kernel size is 2, the step size is 2, and the padding mode is "valid"; the input of NetE is the output DT1 of NetF, and the output is the N×512 secondary feature vector DT2;

[0133] DT2=NetF(DT1)=NetF1(DT1)⊕NetF2(DT2)

[0134] Among them, ⊕ is the sum of the eigenvectors in element order;

[0135] The high-dimensional feature fusion module NetG improves the spatial resolution of features by upsampling and fusion; NetG includes three deconvolution layers, the number of convolution kernels are 256, 128, and 64 respectively, the convolution kernel size is 3, the step size is 2, the padding mode is "same", and the activation function uses ReLU; the input of NetG is the output DT2 of NetF, and the output is the feature secondary feature vector DT3 of N×64;

[0136] DT3=NetG(DT2)

[0137] The classification output head module NetH is used to generate the final classification result; NetH contains two layers of networks, the first layer is a convolution layer, the number of convolution kernels is 64, the convolution kernel size is 1, the step size is 1, the padding mode is "same", and the activation function uses ReLU; the second layer is a convolution layer, the number of convolution kernels is the total number of target object types n, the convolution kernel size is 1, the step size is 1, the padding mode is "same", and the SoftMax activation function is used; the input of NetH is the output DT3 of NetG, and the output is R_cls, which is the classification probability of a single target point cloud R_seg_part in each category;

[0138] R_cls=NetH(DT3).

[0139] The step S3 constructs a target posture recognition model based on point cloud data, and the specific structure is:

[0140] The target posture recognition model includes a feature extraction module NetI and a posture recognition output head module NetJ; the number of times the target posture recognition model runs is equal to the number of target objects Fn recognized by the scene semantic segmentation model;

[0141] The feature extraction module NetI includes a 6-layer network, which is specially designed to extract key features from point cloud data; the 1st, 3rd and 5th layers are convolution layers, the number of convolution kernels are 8, 16 and 32 respectively, the convolution kernel size is 3, the step size is 1, the padding mode is "same", and the activation function uses ReLU; the 2nd, 4th and 6th layers are pooling layers, which use the maximum pooling mode, the pooling kernel size is 2, the step size is 2, and the padding mode is "same"; the input of NetI is Ns×3 point cloud data, and the output is Ns×32 posture feature vector ZT1, where Ns is the number of points in R_seg_part;

[0142] ZT1=NetI(R_seg_part)

[0143] The posture recognition output head module NetJ is used to generate the final posture recognition result; NetJ contains two layers of networks, the first layer is a convolution layer, the number of convolution kernels is 64, the convolution kernel size is 1, the step size is 1, the filling method is "same", and the activation function uses ReLU; the second layer is a fully connected layer, and the output is a 6-dimensional vector loc_ges, the first three dimensions represent the position, and the last three dimensions represent the posture;

[0144] loc_ges=NetJ(ZT1).

[0145] Step S4: Use the target object database D and scene database Dc constructed by S1 and S2 to train the scene semantic segmentation model, target category discrimination model and target posture recognition model constructed by S3 in sequence;

[0146] The scene semantic segmentation model, target type discrimination model and target posture recognition model of step S4 are trained in sequence, specifically including the following three steps:

[0147] Step S401, training the semantic segmentation model, using the scene database Dc constructed in step S2 as training data, training the semantic segmentation model, and using the cross entropy loss as the loss function of the training process. The specific calculation formula is:

[0148]

[0149] Among them, Num is the number of points in the point cloud, Cas is the number of categories, r ij is the one-hot encoding of the semantic label Lb in the scene database flattened into one row, indicating that the i-th point belongs to the j-th category, t ij is the prediction result output by the deep learning model. The value represents the probability that the i-th point belongs to the j-th category. The meanings of i and j are the same as those of r. ij The meaning is the same; the optimizer of the training process uses the Adam optimizer, and the learning rate α1 ranges from:

[0150] α1∈[0.0001,0.01].

[0151] Step S402, training a target type discrimination model, includes:

[0152] Step B1, data enhancement of the target object database D, including random cropping, rotation, and scaling, where the random cropping, rotation, and scaling operations are all performed on the point cloud data P, and finally an enhanced target object database D2 with n2 point cloud data P is obtained, where n2>>n, and the value range of n2 is [100, 1000]. 90% of the obtained target object database D2 is used as training data KD_train, and 10% is used as test data KD_test.

[0153] KD=Data_aug(D2)

[0154] =Data_aug({P,C})

[0155] ={Data_aug(P),Data_aug(C)}

[0156] KD_train,KD_test=random_split(KD,0.9,0.1)

[0157] Among them, Data_aug() represents random cropping, flipping, rotation, and scaling of the point cloud, and random_split() represents the proportional division of the data set, and the proportion is represented by the last two elements;

[0158] Step B2: Use KD_train to train the target category discrimination model. The loss function of the training process uses the cross entropy loss. The specific calculation formula is:

[0159]

[0160] Among them, n2 is the number of data in the target object database D2, n is the number of target object types, r qw The qth element in the target object database D2 belongs to the wth category, t qw is the prediction result output by the target type discrimination model, t qw ∈[0,1], the value represents the probability of the existence or non-existence of scratches, and the meanings of q and w are the same as those of r qw The meaning is the same; the optimizer of the training process uses the Adam optimizer, and the learning rate α2 range is:

[0161] α2∈[0.0001,0.01].

[0162] Step S403, training the target posture recognition model, using the scene database Dc constructed in step S2 as training data, to train the target posture recognition model. The training process uses the L1 loss function for the position information training and the L2 loss function for the posture information training. The specific calculation formula is:

[0163] L1=|loc_ges[:3]-pos|

[0164]

[0165] Among them, loc_ges is the 6-dimensional vector output by the target posture recognition model, pos is the position information corresponding to R_seg_part in the scene database, and ges is the posture information corresponding to R_seg_part in the scene database; the optimizer of the training process uses the SGD optimizer, and the learning rate α3 range is:

[0166] α3∈[0.0001,0.01].

[0167] Step S5: Use a depth camera to collect scene 3D point cloud data of a multi-object stacking scene to be identified, and use the three models trained in S4 to perform recognition tests on the scene that requires target recognition. If the test requirements are met, save the model data; if the test requirements are not met, return to S4 for retraining;

[0168] The depth camera is used to collect depth images of target objects or scenes, and has the ability to transmit the images to a data processing server in real time.

[0169] In step S5, the three models trained in S4 are used to perform recognition tests on scenes requiring target recognition. If the test requirements are met, the model data is saved. If the test requirements are not met, the model data is returned to S4 for retraining. The scene semantic segmentation model uses the mIOU evaluation index, the target type discrimination model uses the global accuracy rate acc evaluation index, and the posture recognition model uses the posture error err evaluation index:

[0170] err=L1+L2

[0171] Among them, L1 is the loss function for position information, and L2 is the loss function for posture information.

[0172] The trained deep learning model can be used for target recognition and estimation of position and posture information.

[0173] Embodiment 2:

[0174] The present invention also provides a robot target recognition system based on deep learning, comprising:

[0175] Depth camera: used to obtain depth images of target objects and scenes, and internal and external parameters must be known in advance;

[0176] Industrial computer: used to obtain depth images and convert them into three-dimensional point cloud information, train three deep learning models provided by the robot target recognition method based on the deep learning method and store model data, test the three deep learning models, and drive the robot to perform operations;

[0177] Robot: used to carry depth cameras and industrial computers.

[0178] To realize the above-mentioned industrial robot target recognition method based on deep learning.

[0179] In detail, the robot target recognition system based on deep learning in the embodiment of the present invention adopts the same method as above when used. Figure 1 The same technical means as the robot target recognition method based on deep learning described in the text can produce the same technical effects, so I will not go into details here.

[0180] It should be noted that the serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages and disadvantages of the embodiments. And the terms "including", "comprising" or any other variants thereof in this article are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, an element defined by the sentence "including a ..." does not exclude the presence of other identical elements in the process, device, article or method including the element.

[0181] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0182] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for industrial robot target recognition based on deep learning, characterized in that: The following steps are involved: Step S1: Use a depth camera to collect a depth image of the target object to be identified, and convert it into three-dimensional point cloud data of the target object, mark the type of the target object, and build a target object database D; Step S2: Use a depth camera to collect multiple known multi-object stacked scene depth images, and convert them into scene three-dimensional point cloud data, mark the position and posture of the target object to be identified in the scene, and build a scene database Dc; Step S3: constructing a scene semantic segmentation model based on the scene 3D point cloud data, constructing a target type discrimination model based on the target object 3D point cloud data, and constructing a target posture recognition model based on the scene 3D point cloud data; The scene semantic segmentation model based on point cloud data is constructed, and the specific structure is as follows: The scene semantic segmentation model includes a point cloud feature extraction module NetA, a multi-scale grouping module NetB, a feature fusion module NetC and a segmentation output head module NetOut; The point cloud feature extraction module NetA includes multiple point cloud feature extraction submodules, and each submodule includes a point cloud sampling layer and two convolutional layers connected in series; the convolutional layer adopts an MLP structure, the MLP dimensions of the first convolutional layer are 64, 64, 128, and the MLP dimensions of the second convolutional layer are 128, 128, 256; the point cloud feature extraction module NetA inputs the scene three-dimensional point cloud data Po of Num x3, and outputs a high-dimensional feature vector set GT1 of Ke x 256, where Num is the number of points in the scene three-dimensional point cloud, and Ke is the number of key points after sampling; GT1=NetA(Po) The multi-scale grouping module NetB includes multiple parallel sub-modules, and each sub-module corresponds to a neighborhood radius of a scale; each sub-module includes a sampling layer, a spherical query layer and two concatenated convolutional layers; the convolutional layer adopts an MLP structure, the MLP dimensions of the first convolutional layer are 128, 128, 256, and the MLP dimensions of the second convolutional layer are 256, 256, 512; the multi-scale grouping module NetB inputs a feature vector set GT1 of Ke x 256, and outputs a multi-scale feature vector set GT2 of Ke x 512; GT2=NetB(GT1) The feature fusion module NetC is a point-by-point feature fusion network, comprising multiple convolutional layers connected in series, and the MLP dimensions of each convolutional layer are 256, 256, and 512; the input of the feature fusion module NetC is a multi-scale feature vector set GT2 of Ke x 512, and the output is a fusion feature vector set GT3 of Ke x 512; GT3=NetC(GT2) The segmentation output head module NetOut contains multiple convolution layers, the MLP dimensions of each convolution layer are 512, 256, 128, and the last layer is a 1x1 convolution layer, which is used to map the feature vector to the category of each point; the segmentation output head module NetOut input is a fusion feature vector set of Ke x 512, and the output is a classification result R_seg of Num x Cas, where Num is the number of points in the scene 3D point cloud, and Cas is the number of scene semantic segmentation categories. The specific calculation method is: R_seg=NetOut(GT3) Cas=n+1 Where n is the total number of target objects; The target type discrimination model based on the three-dimensional point cloud data of the object is constructed, and the specific structure is: The target type discrimination model includes a feature extraction module NetE, a low-dimensional feature fusion module NetF, a high-dimensional feature fusion module NetG and a classification output head module NetH; the number of times the target discrimination model runs is equal to the number of target objects Fn recognized by the scene semantic segmentation model; The feature extraction module NetE includes a six-layer network in series, which is used to preliminarily extract the features of point cloud data; the 1st, 3rd and 5th layers are convolution layers, the number of convolution kernels are 8, 16 and 32 respectively, the convolution kernel size is 3, the step size is 1, the padding mode is "same", and the activation function uses ReLU; the 2nd, 4th and 6th layers are pooling layers, using maximum pooling, the pooling kernel size is 2x2, the step size is 2, and the padding mode is "same"; the input of NetE is a single target point cloud R_seg_part in the semantic segmentation result output by the scene semantic segmentation model, which is Ns×3 point cloud data, and the output is a primary feature vector DT1 of Ns×32, where Ns is the number of points in R_seg_part; R_seg_part∈R_seg DT1=NetE(R_seg_part) The low-dimensional feature fusion module NetF further fuses and processes the point cloud features; NetF includes two parallel feature fusion subnetworks NetF1 and NetF2; each subnetwork contains 10 layers of networks with the following structure: the 1st, 3rd, 5th, 7th, and 9th layers are convolution layers, the number of convolution kernels are 32, 64, 128, 256, and 512 respectively, the convolution kernel size is 1, the step size is 1, the padding mode is "same", and the activation function uses ReLU; the 2nd, 4th, 6th, 8th, and 10th layers are pooling layers, using maximum pooling, the pooling kernel size is 2, the step size is 2, and the padding mode is "valid"; the input of NetE is the output DT1 of NetF, and the output is the N×512 secondary feature vector DT2; in, Sum the eigenvectors in element order; The high-dimensional feature fusion module NetG improves the spatial resolution of features by upsampling and fusion; NetG includes three deconvolution layers, the number of convolution kernels are 256, 128, and 64 respectively, the convolution kernel size is 3, the step size is 2, the padding mode is "same", and the activation function uses ReLU; the input of NetG is the output DT2 of NetF, and the output is the feature secondary feature vector DT3 of N×64; DT3=NetG(DT2) The classification output head module NetH is used to generate the final classification result; NetH contains two layers of networks, the first layer is a convolution layer, the number of convolution kernels is 64, the convolution kernel size is 1, the step size is 1, the padding mode is "same", and the activation function uses ReLU; the second layer is a convolution layer, the number of convolution kernels is the total number of target object types n, the convolution kernel size is 1, the step size is 1, the padding mode is "same", and the SoftMax activation function is used; the input of NetH is the output DT3 of NetG, and the output is R_cls, which is the classification probability of a single target point cloud R_seg_part in each category; R_cls = NetH(DT3); The target posture recognition model based on scene point cloud data is constructed, and the specific structure is as follows: The target posture recognition model includes a feature extraction module NetI and a posture recognition output head module NetJ; the number of times the target posture recognition model runs is equal to the number of target objects Fn recognized by the scene semantic segmentation model; The feature extraction module NetI includes a 6-layer network, which is specially designed to extract key features from point cloud data; the 1st, 3rd and 5th layers are convolution layers, the number of convolution kernels are 8, 16 and 32 respectively, the convolution kernel size is 3, the step size is 1, the padding mode is "same", and the activation function uses ReLU; the 2nd, 4th and 6th layers are pooling layers, which use the maximum pooling mode, the pooling kernel size is 2, the step size is 2, and the padding mode is "same"; the input of NetI is Ns×3 point cloud data, and the output is Ns×32 posture feature vector ZT1, where Ns is the number of points in R_seg_part; ZT1=NetI(R_seg_part) The posture recognition output head module NetJ is used to generate the final posture recognition result; NetJ contains two layers of networks, the first layer is a convolution layer, the number of convolution kernels is 64, the convolution kernel size is 1, the step size is 1, the filling method is "same", and the activation function uses ReLU; the second layer is a fully connected layer, and the output is a 6-dimensional vector loc_ges, the first three dimensions represent the position, and the last three dimensions represent the posture; loc_ges = NetJ(ZT1); Step S4: Use the target object database D and scene database Dc constructed by S1 and S2 to train the scene semantic segmentation model, target category discrimination model and target posture recognition model constructed by S3 in sequence; Step S5: Use a depth camera to collect scene 3D point cloud data of the multi-object stacking scene to be identified, and use the three models trained in S4 to perform recognition tests on the scenes that require target recognition. If the test requirements are met, save the model data; if the test requirements are not met, return to S4 for retraining.

2. The industrial robot target recognition method based on deep learning according to claim 1, characterized in that: The process of converting the depth image of the target object to be identified in step S1 into three-dimensional point cloud data is to convert the two-dimensional coordinates and depth value of each pixel point of the depth image of the target object to be identified into the three-dimensional coordinates and color value of the point cloud data. The internal parameters of the depth camera are represented by a 3x3 matrix N: Among them, j x is the horizontal focal length of the camera, j y is the vertical focal length of the camera, z x is the horizontal coordinate of the camera's principal point, z y is the ordinate of the principal point of the camera; the intrinsic parameter matrix N of the depth camera converts the image coordinates of a two-dimensional point on a depth image into the coordinates of a three-dimensional point cloud data point of an object in the camera coordinate system. The formula is as follows: Among them, sx, sy, sz are the specific coordinate information of the three-dimensional point cloud in the depth camera coordinate system, eu, ev are the coordinate information of the two-dimensional depth image, ez is the depth information of the two-dimensional depth image at the position of eu, ev, N -1 is the inverse matrix of the camera's intrinsic parameter matrix N; the depth camera's extrinsic parameters are represented by a 4x4 matrix W: Among them, X is a 3x3 rotation matrix, q is a 3x1 translation vector; the camera's external parameter matrix W converts the three-dimensional point coordinates in a camera coordinate system into a three-dimensional point coordinates in a world coordinate system. The formula is as follows: Among them, X -1 is the inverse matrix of the rotation matrix X, gx, gy, and gz are the specific coordinate information of the three-dimensional point cloud in the world coordinate system.

3. The industrial robot target recognition method based on deep learning as claimed in claim 2, characterized in that: In the step S1, the object types are marked and the target object database is constructed by establishing a correspondence between the three-dimensional point cloud data P of the object in the world coordinate system and the actual category C of the object, and obtaining the target object database D. The target database D contains n groups of data, where n is the total number of target object types; D={[P1,C1],[P2,C2],...,[P n ,C n ]} Among them, all subscripts refer to the order, P1…P n is the 3D point cloud data of different objects, C1…C n is the actual category C of the object point cloud data.

4. The industrial robot target recognition method based on deep learning as claimed in claim 2, characterized in that: In the step S2, the position and posture of the target object to be identified in the scene are marked, and the scene database is constructed by establishing the correspondence between the scene three-dimensional point cloud data Po in the world coordinate system, the scene semantic label Lb, and the type C, position pos, and posture ges of all target objects contained in the scene, and obtaining the scene database Dc, Dc contains m groups of data, and m is the number of scenes collected; Among them, Po1…Po m Three-dimensional point cloud data Po, Lb1…Lb for different scenes m are the corresponding scene semantic labels Lb, C m1 …C mk Po for scene 3D point cloud data m The actual category C of each object in pos m1 …pos mk Po for scene 3D point cloud data m The position of each object in pos,ges m1 …ges mk Po for scene 3D point cloud data m The posture of each object in ges.

5. The industrial robot target recognition method based on deep learning according to claim 1, characterized in that: The scene semantic segmentation model, target type discrimination model and target posture recognition model of step S4 are trained in sequence, specifically including the following three steps: Step S401, training the semantic segmentation model, using the scene database Dc constructed in step S2 as training data, training the semantic segmentation model, and using the cross entropy loss as the loss function of the training process. The specific calculation formula is: Among them, Num is the number of points in the point cloud, Cas is the number of categories, r ij is the one-hot encoding of the semantic label Lb in the scene database flattened into one row, indicating that the i-th point belongs to the j-th category, t ij is the prediction result output by the deep learning model. The value represents the probability that the i-th point belongs to the j-th category. The meanings of i and j are the same as those of r. ij The meaning is the same; the optimizer of the training process uses the Adam optimizer, and the learning rate α1 ranges from: α1∈[0.0001,0.01] Step S402, training a target type discrimination model, includes: Step B1, data enhancement of the target object database D, including random cropping, rotation, and scaling, where the random cropping, rotation, and scaling operations are all performed on the point cloud data P, and finally an enhanced target object database D2 with n2 point cloud data P is obtained, where n2>>n, and the value range of n2 is [100, 1000]. 90% of the obtained target object database D2 is used as training data KD_train, and 10% is used as test data KD_test. KD=Data_aug(D2) =Data_aug({P,C}) ={Data_aug(P),Data_aug(C)} KD_train,KD_test=random_split(KD,0.9,0.1) Among them, Data_aug(·) represents random cropping, flipping, rotation, and scaling of the point cloud, and random_split(·) represents the proportional division of the data set, and the proportion is represented by the last two elements; Step B2: Use KD_train to train the target category discrimination model. The loss function of the training process uses the cross entropy loss. The specific calculation formula is: Among them, n2 is the number of data in the target object database D2, n is the number of target object types, r qw The qth element in the target object database D2 belongs to the wth category, t qw is the prediction result output by the target type discrimination model, t qw ∈[0,1], the value represents the probability of the existence or non-existence of scratches, and the meanings of q and w are the same as those of r qw The meaning is the same; the optimizer of the training process uses the Adam optimizer, and the learning rate α2 range is: α2∈[0.0001,0.01] Step S403, training the target posture recognition model, using the scene database Dc constructed in step S2 as training data, to train the target posture recognition model. The training process uses the L1 loss function for the position information training and the L2 loss function for the posture information training. The specific calculation formula is: L1=|loc_ges[:3]-pos| Among them, loc_ges is the 6-dimensional vector output by the target posture recognition model, pos is the position information corresponding to R_seg_part in the scene database, and ges is the posture information corresponding to R_seg_part in the scene database; the optimizer of the training process uses the SGD optimizer, and the learning rate α3 range is: α3∈[0.0001,0.01]。 6. An industrial robot target recognition system based on deep learning, characterized in that: include: Depth camera: used to obtain depth images of target objects and scenes, and internal and external parameters must be known in advance; Industrial computer: used to obtain depth images and convert them into three-dimensional point cloud information, train three deep learning models provided by the robot target recognition method based on the deep learning method and store model data, test the three deep learning models, and drive the robot to perform operations; Robot: used to carry depth cameras and industrial computers; To realize an industrial robot target recognition method based on deep learning as described in any one of claims 1-5.

Images