Industrial robot posture recognition method

Abstract

The invention relates to the technical field of industrial automation, in particular to an industrial robot posture recognition method. Aiming at the problems of weak workpiece texture, strong surface reflection, serious stacking and shielding and the like in an industrial production line, a pixel-level dense feature fusion and self-attention mechanism is introduced, and semantic texture information of a color image and spatial geometric information of a depth image are integrated in a feature extraction stage; the problem that the posture of a rotationally symmetrical workpiece is fuzzy is solved through an asymmetric loss function, meanwhile, a comprehensive grabbing scoring model containing force sealing stability, environment collision risk probability, mechanical arm kinematics reachability and visual uncertainty is further established, and through the mode of combining off-line grabbing candidate construction and on-line real-time evaluation, the grabbing accuracy of the mechanical arm is improved. And the globally optimal grabbing pose is screened out. According to the method, the recognition precision and robustness of the industrial robot in the unstructured environment are remarkably improved, and the success rate and safety of grabbing operation are improved.

Description

Technical Field

[0001] This invention relates to the field of industrial automation technology, specifically to a method for industrial robot posture recognition. Background Technology

[0002] With the deepening of the intelligent manufacturing strategy, industrial robots have transformed from traditional teach-and-reproduce robots to intelligent robots with environmental perception and autonomous decision-making capabilities. In this transformation process, the robot's "hand-eye coordination" ability—that is, recognizing the position of objects through visual sensors and controlling the robotic arm to perform precise operations—has become a core technological bottleneck.

[0003] In traditional industrial applications, robots typically operate in structured environments where the arrival position and orientation of objects are fixed (ensuring this through positioning fixtures). However, modern flexible manufacturing requires robots to handle randomly stacked parts in bins or randomly flowing workpieces on conveyor belts. This necessitates that vision systems be able to accurately estimate the 6-DOF pose of objects in three-dimensional space, namely position and orientation.

[0004] Existing technical solutions to this problem can be mainly divided into: 2D template matching-based methods: These methods locate objects through edge detection or grayscale correlation matching. While computationally fast, they can only acquire position and rotation within a plane, failing to perceive depth information or handle rotations along non-optical axes. For tilted or stacked objects, this method completely fails.

[0005] 3D methods based on artificial features, such as matching using Fast Point Feature Histograms (FPFH) or SHOT features from point clouds, rely heavily on the geometric richness of the object's surface. However, industrial parts are mostly composed of regular planes and cylinders, lacking significant geometric features, resulting in low discriminative power of feature descriptors and a high mismatch rate.

[0006] Traditional RGB feature-based methods rely on the texture information of the object's surface. However, industrial parts are mostly metal or injection molded parts with smooth, uniform surfaces (weak texture) and are prone to specular reflection (high reflectivity), making feature point extraction difficult or unstable.

[0007] Early deep learning methods: These methods typically detect 2D bounding boxes first, and then calculate the pose using the PnP algorithm. These methods are limited by the accuracy of the 2D bounding boxes, and in cases of severe occlusion, the bounding boxes contain a large amount of background noise, which seriously affects the accuracy of the pose calculation.

[0008] In summary, single visual recognition or single motion planning can no longer meet the high reliability requirements of modern industry. There is a need for a systematic method that can deeply integrate RGB and deep multimodal information to improve recognition robustness and dynamically optimize grasping posture in combination with physical environment constraints. Summary of the Invention

[0009] To address the shortcomings of existing technologies, this invention provides an industrial robot posture recognition method.

[0010] An industrial robot posture recognition method includes the following steps: RGB-D depth cameras installed at the end effector or fixed position of industrial robots are used to acquire color and depth images of the target work area; Distortion correction of color images is performed using the camera intrinsic parameter matrix; Based on the hand-eye transformation matrix, depth images are converted into point cloud data and aligned with color images at the pixel level; The aligned color image is input into a pre-trained 2D instance segmentation network, which outputs image patches and pixel-level masks of various target objects in the scene. The depth patch corresponding to the target object is cropped from the depth image based on the mask, and the three-dimensional point cloud cluster of the target object is generated by back-projection using the camera intrinsic parameters. Based on the image patches and 3D point cloud clusters of the target object, the pose of the target object at future moments is predicted using a pre-trained deep neural network. Using the predicted future pose of the object as the initial value, the pose is optimized using an iterative refinement network to obtain a high-precision 6D pose of the target. In advance, in an offline simulation environment, based on the 3D CAD model of the target object, a dense set of grabbing candidate poses covering the surface of the object is sampled and generated. Using the target 6D location data, all grabbing postures in the candidate posture set are transformed to the world coordinate system to generate a scene feasible grabbing candidate pool; Construct a comprehensive scoring function to quantitatively evaluate each candidate pose in the feasible crawling candidate pool for the scene; The candidate poses are sorted according to the scoring results, and the pose with the highest score is selected as the optimal grasping pose.

[0011] Preferably, the step of predicting the future pose of the target object using a pre-trained deep neural network based on image patches and 3D point cloud clusters of the target object specifically includes: Input the image patch of the target object into the color feature extraction branch to extract the color feature vector of the target object; Input the 3D point cloud cluster of the target object into the geometric feature extraction branch to extract the geometric feature vector of each point; The color feature vector and the geometric feature vector are concatenated and fused to generate a fused feature vector that includes local details and global context; The fused feature vector is input into the pose prediction head to predict the pose of the target object at future moments.

[0012] Preferably, the color extraction branch includes a downsampling encoding path and an upsampling decoding path, and fuses the deep semantic features and shallow texture features of the image patch of the target object through skip connections, outputting a feature map with the same spatial resolution as the input image, wherein each pixel position corresponds to a color feature vector.

[0013] Preferably, the step of inputting the 3D point cloud cluster of the target object into the geometric feature extraction branch and extracting the geometric feature vector of each point specifically includes: The input point cloud cluster is normalized and rotated using a spatial transformation network; The coordinates of each point are mapped to a high-dimensional feature space by a multilayer perceptron with shared weights, generating point-level local geometric features; Max pooling is used to aggregate the features of all points to generate a global geometric feature vector that reflects the overall topology of the object; The global geometric feature vector is concatenated with the point-level local geometric features of each point to output the geometric feature vector corresponding to each point.

[0014] Preferably, the step of concatenating and fusing the color feature vector with the geometric feature vector to generate a fused feature vector containing local details and global context specifically includes: The geometric feature vector and the color feature vector are concatenated along the feature channel dimension to form an initial fused feature vector; Max pooling is performed on the local geometric feature vectors of all input points to extract the global geometric feature vectors that reflect the overall shape information of the object. The global geometric feature vector is concatenated with the initial mixed feature vector of each point to form an augmented feature vector containing both local and global information. The augmented feature vector is input into a shared multilayer perceptron consisting of fully connected layers, batch normalization layers, and nonlinear activation functions to learn the nonlinear interaction between color semantics and spatial geometry, and outputs the final fused feature vector for each point.

[0015] Preferably, the step of optimizing the pose using an iterative refinement network to obtain a high-precision target 6D pose specifically includes: Obtain the current attitude estimate and construct its inverse transformation matrix; Using the inverse transformation matrix, the input point cloud is transformed to the current normal coordinate system to generate the transformed point cloud; The transformed point cloud is input into the point cloud encoder network to extract new local geometric features. The new local geometric features are then concatenated with the color feature vector to form a comprehensive feature vector containing the current pose error information. The comprehensive and refined feature vector is input into the attitude residual prediction network, and the attitude correction is output. The current attitude is updated using the attitude correction amount to obtain the updated attitude estimate; The updated attitude estimate is used as the input for the next iteration. The above steps are repeated until the preset number of iterations is reached or the attitude correction is less than the preset threshold. The final optimized 6D position data is then output.

[0016] Preferably, the deep neural network training process employs an adaptive loss function tailored to the geometric characteristics of industrial parts: For asymmetric objects, average distance loss is used; For rotationally symmetric or axisymmetric objects, the average nearest point distance loss is used.

[0017] Compared with the prior art, the advantages of this invention are: A pixel-level dense feature fusion scheme is adopted. By deeply fusing the color texture features of the RGB image with the geometric structure features of the depth point cloud at each pixel, the network can still make effective inferences based on geometric features even when objects are partially occluded or texture features are not obvious; conversely, when geometric features are missing (such as for planar objects), texture features are used to supplement them. This complementary mechanism ensures the high reliability of the system in environments with large changes in lighting and cluttered backgrounds.

[0018] A deep learning-based iterative refinement module is introduced. This module corrects errors by inversely transforming the currently estimated pose to a canonical coordinate system and predicting the pose residuals. Compared to traditional algorithms, this method utilizes the semantic information of the image to assist geometric registration, achieving not only fast convergence but also sub-millimeter-level positioning accuracy, thus balancing accuracy and real-time performance.

[0019] A multi-dimensional comprehensive scoring model was constructed, incorporating force-closed stability, environmental collision risk, and visual confidence. By dynamically selecting the highest-scoring posture from the grasping candidate pool, not only was the mechanical stability of the grasping guaranteed, but environmental obstacles and the robot arm's motion dead zones were also avoided, significantly reducing the risk of production line downtime. Attached Figure Description

[0020] Figure 1 This is a flowchart of an industrial robot posture recognition method proposed in this invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0022] Reference Figure 1 This invention provides a method for industrial robot posture recognition.

[0023] S1. Acquire color and depth images of the target work area using an RGB-D depth camera installed at the end of an industrial robot or at a fixed position: Because industrial camera lenses (especially wide-angle or fisheye lenses) have optical distortion, straight lines at the edges of the image become curved, directly affecting the accuracy of subsequent pose estimation. Therefore, distortion correction of color images is necessary. This embodiment adopts a distortion correction method based on the Brown-Conrady model, and the specific steps are as follows: Distortion parameter acquisition: During the calibration phase, multiple images of the checkerboard calibration board from different angles were captured using a camera.

[0024] The coordinates of the interior corners in each image are extracted using a corner detection algorithm, and the camera's intrinsic parameter matrix is ​​calculated using the Zhang Zhengyou calibration method. and distortion coefficient vector .in The radial distortion coefficient is... denoted as the tangential distortion coefficient.

[0025] Construct a correction mapping table: For each pixel in the output corrected image First, the points are transformed to the normalized image coordinate system using the inverse of the intrinsic parameter matrix. : ; Calculate the square of the distance from the point to the optical center. .

[0026] The Brown-Conrady distortion model was used to calculate the coordinates of the original distorted image. : ; Will Projecting back into the pixel coordinate system yields the floating-point coordinates in the original distorted image. : ; Resampling and Generation: Using bilinear interpolation, based on the calculated floating-point coordinates Extract grayscale or RGB values ​​from the original image and assign them to points in the correction image. This generates a distortion-free corrected image.

[0027] Accurate image calibration is fundamental to all subsequent operations. This embodiment employs an automatic calibration process based on a checkerboard pattern.

[0028] Data acquisition sequence: A robot program was written to move its end effector around the center of the calibration plate, moving in different positions and postures on the hemispherical surface, collecting a total of [data acquisition data]. Groups of data. Each group of data includes: Transformation matrix from robot base to end effector (Read directly from the controller).

[0029] An image captured by a camera that contains a checkerboard pattern.

[0030] Image pose calculation: Using OpenCV's findChessboardCorners and solvePnP functions, the transformation matrix of the calibration board relative to the camera is calculated. .

[0031] Equation construction: For any two sets of poses and Since the calibration plate is fixed in the world coordinate system and the relative relationship between the camera and the flange remains constant, the following constraints exist: ; in The hand-eye matrix is ​​to be determined.

[0032] Solution and Optimization: The Tsai-Lenz algorithm is used to solve the above constraint formula to obtain analytical solutions as initial values.

[0033] Based on the aforementioned hand-eye transformation matrix, the depth image is converted into point cloud data and aligned with the color image at the pixel level.

[0034] The aligned color image is input into a pre-trained 2D instance segmentation network, which outputs image patches and pixel-level masks of various target objects in the scene.

[0035] The depth patch corresponding to the target object is cropped from the depth image based on the mask, and the three-dimensional point cloud cluster of the target object is generated by back-projection using camera intrinsic parameters.

[0036] S2. Based on the image blocks and 3D point cloud clusters of the target object, a pre-trained deep neural network is used to predict the pose of the target object at future moments.

[0037] The deep neural network receives: RGB image patch: size scaled to .

[0038] Point cloud: random sampling There are 1 point, each containing coordinates. .

[0039] Heterogeneous features are extracted through two parallel branches: RGB color feature extraction branch: The modified ResNet-18 is used as the backbone network, which is essentially a fully convolutional network.

[0040] Encoder: Input The image is downsampled stepwise through four residual modules to extract high-level semantic information.

[0041] Decoder: In order to avoid losing spatial resolution (to facilitate subsequent one-to-one correspondence with point clouds), the network contains two upsampling layers, which fuse shallow texture features through bilinear interpolation and skip connections.

[0042] Output: The final output feature map size is restored to... That is, each pixel location has a 32-dimensional semantic feature vector.

[0043] Geometric feature extraction branch: Input is A chaotic cluster of points.

[0044] Point feature transformation: First, a spatial transformation network is used to perform rigid body transformation correction on the input point cloud to make it invariant to input rotation.

[0045] MLP mapping: Each point is processed using a series of multilayer perceptrons with shared weights to generate high-dimensional local features.

[0046] Global aggregation: The max pooling operation is used to aggregate the information of all points to obtain a 1024-dimensional global geometric feature vector.

[0047] Feature concatenation: Copying global features Next, after splicing the local features of each point, further MLP dimensionality reduction is performed to output the geometric feature vector of each point.

[0048] Geometric projection and indexing: For each point in the point cloud input The camera intrinsic parameter matrix is ​​used to project it back onto the 2D image plane: ; Due to floating-point coordinates It doesn't have to be an integer. Bilinear interpolation is used to sample the feature map of the color image to obtain the color feature vector corresponding to that point.

[0049] Feature splicing: The sampled color features are combined with the geometric features of the point to obtain a hybrid feature.

[0050] Global context enhancement: To enable each point to perceive the overall shape of the object, max pooling is performed on the geometric features of all points.

[0051] Copy the geometric features after max pooling. The augmented features are obtained by combining the features of each point with the mixed features of each point.

[0052] Nonlinear fusion: The augmented features are input into a shared 3-layer MLP network (layer widths of 512, 256, and 128), with each layer followed by a ReLU activation function. This network learns how to adaptively combine local texture and geometric information based on the global context, ultimately outputting fused features.

[0053] For each fused point feature, the deep learning network outputs three predicted values ​​in parallel: Translation offset: Predicts the position of the target center relative to the current point.

[0054] Rotation quaternions: Predict the rotational orientation of an object.

[0055] Confidence level: A value that indicates the reliability of the prediction result at this point.

[0056] Finally, the set of rotation quaternions with the highest confidence is selected as the initial rotation, and the translation offset with the highest confidence is selected as the basis for determining the predicted target center position.

[0057] It should be noted that the loss function of this neural network is: ; in Choose either the average distance loss function or the average nearest point distance loss function based on the object's symmetry. For confidence regularization, prevent the network from simply setting all confidence levels to 1 or 0: ; In this way, if the attitude prediction error at a certain point is... The value is very large; to reduce the overall loss, the network will automatically lower the confidence level at that point. ;but This also forces the confidence level to not be too low.

[0058] S3. Using the predicted future pose of the object as the initial value, the pose is optimized using an iterative refinement network to obtain a high-precision 6D pose of the target, specifically including: In the In each iteration, for each input point, based on the currently estimated rotation quaternion... Translation offset Calculate its coordinates in the normal coordinate system: ; In the formula, For input points Coordinates in a normalized coordinate system.

[0059] This step eliminates the influence of the object's absolute position in the world coordinate system, allowing the network to focus on learning relative errors.

[0060] Transformed point cloud The input is fed into a lightweight PointNet network to extract new geometric features. These features are then concatenated with the color features extracted in step S2.

[0061] The concatenated features are input into a fully connected network, which outputs the incremental pose. and .

[0062] The attitude is updated based on the output attitude increment.

[0063] Repeat the above steps until the preset number of iterations or the pose correction amount is less than the preset threshold, and output the final optimized 6D position data.

[0064] S4. In the offline simulation environment, based on the 3D CAD model of the target object, a dense set of grabbing candidates is pre-constructed, specifically including: Surface sampling: Uniformly sampling contact points on the surface of an object.

[0065] Normal alignment: Set the gripping approach direction to be opposite to the surface normal, and generate multiple gripping angles by rotating around the normal every 15 degrees.

[0066] Force-closure screening: Using the GraspIt! simulation environment, each generated candidate pose is simulated for grasping. Pose that meet the force closure condition and have a grasping quality score higher than a preset threshold (e.g., 0.5) are retained to form a dense grasping candidate set.

[0067] S5. Using the target 6D location data, transform all grasping postures in the candidate posture set to the current robot base coordinates to generate a feasible grasping candidate pool for the scene, specifically including: S501, Coordinate System Mapping Transformation: First, using a hand-eye calibration matrix and real-time object pose The grabbing posture of each grabbing candidate in the dense grabbing candidate set is transformed to the world coordinate system, thus obtaining the candidate grabbing pool in the world coordinate system: ; In the formula, For hand-eye calibration matrix, For real-time object location data. For the first The coordinate representation of the grasping posture. For the first The representation of a grabbing posture in the world coordinate system.

[0068] S502. Filter and verify the candidate grabbing pool in the world coordinate system, and remove unusable grabbing postures, including: Geometric feasibility filtering: Workspace limitations: Check if the gripping point is within the robot's maximum working radius and above the worktable.

[0069] Approach Direction Restriction: Calculate the angle between the grab approach vector and the Z-axis of the world coordinate system (gravity direction). If the angle is greater than... If the robot attempts to grab from bottom to top, the posture is discarded to avoid collision with the table or exceeding the limits of the robotic arm joints.

[0070] Environmental collision detection: Using the real-time point cloud data obtained in step S1, an octree map of the environment is constructed to represent the current scene occupancy information.

[0071] The robot's end effector is simplified into a set of oriented bounding boxes, typically consisting of two parts: the "palm" and the "fingers".

[0072] For each candidate pose that has passed the geometric filtering, place its corresponding gripper-oriented bounding box model into the scene, and query whether there is an environmental point cloud within the area of ​​the directed bounding box.

[0073] If the number of environmental points within the bounding box exceeds a threshold (e.g., 5 points), it is determined that a collision has occurred, and the grabbing posture is removed.

[0074] Verification of inverse kinematics: For the remaining collision-free poses, check if there is a set of joint angles that can achieve the end pose.

[0075] If multiple solutions exist, select the one that minimizes the change in the current configuration of the robotic arm.

[0076] If there is no solution, then the pose is discarded.

[0077] S503. Generate the final candidate pool: After the above screening, the remaining set of poses constitutes the candidate pool for feasible scene capture.

[0078] S6. Construct a comprehensive scoring function to quantitatively evaluate each candidate pose in the feasible capture candidate pool for the scene.

[0079] The mathematical expression for the comprehensive scoring function is: ; In the formula, For posture The final score, For posture Force-sealed stability score, for posture Collision risk score, For posture Visual uncertainty score , and For the corresponding weights.

[0080] The force-closed stability score is used to assess the mechanical stability of the gripping point, preventing objects from slipping during high-speed transport. The specific calculation method is as follows: a gripping matrix is ​​constructed based on spiral theory, and the minimum singular value of the gripping matrix is ​​used as the force-closed stability score.

[0081] Collision risk probability scores are used to assess the collision risk of objects in complex scenes. The specific calculation method is as follows: Calculate the Euclidean distance from the center of the robot gripper's bounding box to the nearest point in the environmental point cloud. .

[0082] The collision risk score is calculated based on the Euclidean distance. ; In the formula, The preset collision safety distance threshold, This is the attenuation coefficient.

[0083] Visual uncertainty scoring utilizes the output confidence of a deep learning network to prioritize regions with more reliable visual recognition results for capture, avoiding areas with severe glare, blurriness, or occlusion. Specifically: The captured touch point is projected back onto the image plane to obtain the average confidence level of pixels in the neighborhood of the touch point. .

[0084] Calculate the visual uncertainty score based on the average confidence level of pixels in the neighborhood: ; Finally, the candidate poses are sorted according to the scoring results, and the pose with the highest score is selected as the optimal grasping pose.

[0085] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0086] The preferred embodiments of the present invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention. The invention is limited only by the claims and their full scope and equivalents.

Claims

1. A method for pose recognition of an industrial robot, characterized in that, The method comprises the following steps: acquiring color images and depth images of a target working area by using an RGB-D depth camera mounted at the end of an industrial robot or a fixed position; correcting the color images by using an intrinsic matrix of the camera; converting the depth images into point cloud data based on a hand-eye transformation matrix, and aligning the point cloud data with the color images at a pixel level; inputting the aligned color images into a pre-trained two-dimensional instance segmentation network to output image blocks and pixel-level masks of various target objects in the scene; cropping depth image blocks corresponding to the target objects from the depth images according to the masks, and generating three-dimensional point cloud clusters of the target objects by using camera intrinsic back projection; predicting the poses of the target objects at a future time by using a pre-trained deep neural network based on the image blocks and the three-dimensional point cloud clusters of the target objects; taking the predicted poses of the objects at the future time as initial values, optimizing the poses by using an iterative refinement network, and obtaining high-precision target 6D poses; pre-sampling dense grasping candidate pose sets covering the surfaces of the target objects in an offline simulation environment based on three-dimensional CAD models of the target objects; transforming all grasping poses in the candidate pose sets into a world coordinate system by using the target 6D poses, and generating a scene feasible grasping candidate pool; constructing a comprehensive scoring function to quantitatively evaluate each candidate pose in the scene feasible grasping candidate pool; sorting the candidate poses according to the scoring results, and selecting the pose with the highest score as the optimal grasping pose.

2. The industrial robot pose recognition method of claim 1, wherein, The method of predicting the poses of the target objects at a future time by using a pre-trained deep neural network based on the image blocks and the three-dimensional point cloud clusters of the target objects specifically comprises the following steps: inputting the image blocks of the target objects into a color feature extraction branch to extract color feature vectors of the target objects; inputting the three-dimensional point cloud clusters of the target objects into a geometric feature extraction branch to extract geometric feature vectors of each point; splicing and fusing the color feature vectors and the geometric feature vectors to generate fusion feature vectors containing local details and global context; inputting the fusion feature vectors into a pose prediction head to predict the poses of the target objects at a future time.

3. The method of claim 2, wherein, The color extraction branch comprises a down-sampling encoding path and an up-sampling decoding path, and the deep semantic features and the shallow texture features of the image blocks of the target objects are fused by jump connection to output feature maps consistent with the spatial resolution of the input images, wherein each pixel position corresponds to a color feature vector.

4. The industrial robot pose identification method of claim 2, wherein, The method of inputting the three-dimensional point cloud clusters of the target objects into a geometric feature extraction branch to extract geometric feature vectors of each point specifically comprises the following steps: normalizing and rotating the input point cloud clusters by using a spatial transformation network; mapping the coordinates of each point to a high-dimensional feature space by a multilayer perceptron with shared weights to generate point-level local geometric features; aggregating the features of all points by using a max-pooling operation to generate a global geometric feature vector reflecting the overall topology of the object; splicing the global geometric feature vector and the point-level local geometric features of each point to output the geometric feature vector corresponding to each point.

5. The industrial robot pose recognition method of claim 1, wherein, The color feature vector is spliced with the geometric feature vector to generate a fusion feature vector containing local details and global context, and the method specifically comprises the following steps: The geometric feature vector and the color feature vector are spliced in the feature channel dimension to form an initial fusion feature vector; A maximum pooling operation is performed on the local geometric feature vectors of all input points to extract a global geometric feature vector reflecting the overall shape information of the object; The global geometric feature vector is spliced with the initial mixed feature vector of each point to form an augmented feature vector containing local and global information; The augmented feature vector is input into a shared multi-layer perceptron composed of a fully connected layer, a batch normalization layer and a nonlinear activation function, the nonlinear interaction relationship between color semantics and spatial geometry is learned, and a final fusion feature vector of each point is output.

6. The industrial robot pose identification method of claim 1, wherein, The pose is optimized by using the iterative refinement network to obtain a high-precision target 6D pose, and the method specifically comprises the following steps: An inverse transformation matrix of the current pose estimate is obtained; The input point cloud is transformed into the current canonical coordinate system by using the inverse transformation matrix to generate a transformed point cloud; The transformed point cloud is input into a point cloud encoder network to extract new local geometric features, and the new local geometric features are spliced with the color feature vector to form a comprehensive feature vector containing current pose error information; The comprehensive refinement feature vector is input into a pose residual prediction network to output a pose correction amount; The current pose is updated by using the pose correction amount to obtain an updated pose estimate; The updated pose estimate is taken as the input of the next iteration, and the above steps are repeatedly executed until a preset iteration number is reached or the pose correction amount is less than a preset threshold, and a final optimized 6D pose is output.

7. The industrial robot pose identification method of claim 1, wherein, The deep neural network training process adopts an adaptive loss function for the geometric characteristics of industrial parts: For asymmetric objects, an average distance loss is used; For rotationally symmetric or axially symmetric objects, an average nearest point distance loss is used.

Images