Deep learning-based pose solving method for weak-texture disordered stacked parts

Abstract

The invention relates to a pose solving method for weak-texture disordered stacked parts based on deep learning. Firstly, annotation data for network training is automatically generated through a computer technology, and the cost consumption of manual annotation is greatly reduced. Secondly, attention matrix calculation is added in the aspect of network design, the point cloud instance segmentation effect is improved, an added feature point prediction module predicts single part instance point cloud, predefined feature point positions are solved, and then the network has the part pose solving capacity through point cloud registration. Compared with a traditional point cloud registration method, the feature point registration method provided by the invention greatly reduces the time loss, and avoids the defect of poor algorithm performance caused by different sampling distances of sensors at the same time. Finally, the accurate positions of the feature points are solved through a Hough voting method in the inference stage, errors introduced by instance segmentation are optimized, and the accuracy of pose prediction of the weak-texture disordered stacked parts is further improved.

Description

Technical Field

[0001] The present invention relates to the technical field related to intelligent manufacturing, and in particular to a method for solving the pose of disordered stacked parts with weak textures based on deep learning. Background Art

[0002] At present, in the automated sorting production lines in the industry, such as the production lines for automobile parts, a large number of parts are placed in the material frame when loading, and the parts are mostly scattered and stacked in disorder. The robot automatically places the parts one by one in the designated position before welding and assembling the parts. The mainstream practice in the market is to use RGB cameras or RGBD cameras to solve the homogeneous transformation matrix of the workpiece relative to the robot base through images and point cloud algorithms. For disorderly stacked weak-textured workpieces, it is difficult to design an algorithm to solve the position and posture of the workpiece due to the few surface features of the workpiece and the serious occlusion between the workpieces. At the same time, deep learning is developing rapidly. Although deep learning methods can extract higher-dimensional features of objects, the high cost of training data annotation, high computing power requirements, and weak stability and reliability have led to deep learning still not being widely used in industrial scenarios. Summary of the invention

[0003] The purpose of the present invention is to solve at least one of the deficiencies of the prior art and to provide a method for solving the pose of disordered stacked parts with weak textures based on deep learning.

[0004] In order to achieve the above object, the present invention adopts the following technical solutions:

[0005] Specifically, a pose solving method for disorderly stacked parts with weak textures based on deep learning is proposed, including the following:

[0006] Acquisition of actual point cloud of parts: obtaining scanning point cloud of parts in the scanning plane, segmenting and preprocessing the scanning point cloud to obtain point cloud files of the front and back sides of several parts;

[0007] CAD model point cloud generation: pre-generate a batch of homogeneous transformation matrices of parts relative to the world coordinate system when the parts are stacked, so as to generate training data by reading the point cloud files and transforming them to the corresponding stacking positions based on the homogeneous transformation matrix;

[0008] Deep learning model and training: Train the improved FPCC network based on the training data, and save the weight file of the trained network.

[0009] Specifically, the improved FPCC network adds a feature point prediction module on the basis of the original FPCC network, and adds an attention matrix to the instance segmentation loss calculation of the original FPCC network;

[0010] Actual stacked part pose calculation: obtain the scanned point cloud of the target part, perform point cloud preprocessing on the scanned point cloud of the target part and input it into the weight file for prediction, obtain the output center score, point cloud instance label and feature point coordinates, and perform Hough voting to obtain the feature point position of the target part;

[0011] The homogeneous transformation matrix corresponding to the target part is obtained by performing ICP point cloud registration with the feature point position and the feature point template.

[0012] Based on the homogeneous transformation matrix corresponding to the target part, the workpiece with a registration error lower than a preset threshold is found and its pose is output.

[0013] Further, specifically, scanning a part in a plane to obtain a scanned point cloud, segmenting and preprocessing the scanned point cloud to obtain point cloud files of the front and back sides of several parts, including:

[0014] The scanning point cloud of the part in the plane is collected by a 3D camera, and the point cloud belonging to the part is segmented from the scanning point cloud to obtain the part point cloud. A total of K part point clouds are collected from the front and back sides. At the same time, the CAD model of the part is collected, and the CAD model is sampled into a point cloud. The FPS farthest point sampling algorithm is used to sample K part feature points, and K point cloud files are constructed based on the part point cloud and part feature points.

[0015] Further, specifically, a homogeneous transformation matrix of a batch of parts relative to the world coordinate system is pre-generated when the parts are stacked, including:

[0016] A rectangular material frame is placed in the simulation environment, and a batch of parts with random heights and random positions are generated above the material frame. The batch of parts are made to fall freely into the material frame. The number of parts gradually increases from 3 to 50, and this is repeated 20 times. The homogeneous transformation matrix of each part relative to the world coordinate system when it is stationary is recorded.

[0017] Further, specifically, the process of generating training data includes,

[0018] Read the homogeneous transformation matrix of the saved parts relative to the world coordinate system one by one, randomly select the part point cloud and its corresponding part feature points in the collected point cloud file and transform them to the physical simulation position to obtain the transformed part point cloud and part feature points, remove all the blocked points in the transformed part point cloud and part feature points, assume that a single part point cloud has N points, let the i-th (i≤N) point of the workpiece point cloud be p i (x i ,y i ,z i ), x i ,y i ,z iare the coordinate values ​​of the i-th point in the point cloud, and the center point of the workpiece point cloud is c, p i The Euclidean distance from the point to the center point c is d i , calculate p respectively i The center score s i , point to feature point p k OffsetOf ki ,

[0019]

[0020] Of Ki =p i -p k

[0021] All points in the scene are used as training data according to their coordinates, normalized coordinates, instance labels, center scores, and feature point offsets.

[0022] Further, specifically, the improved FPCC network includes,

[0023] A feature point prediction module is added on the basis of the FPCC network, so that each point can estimate the coordinates of K feature points and then obtain the redundant feature point coordinates. After obtaining the redundant feature point coordinates, the Hough voting method is used to obtain the position of the feature point with the highest number of votes. Finally, the ICP point cloud registration method of the feature points is used to solve the pose of the workpiece. At the same time, the attention matrix is ​​added to the instance segmentation loss calculation of the FPCC network. According to the distance matrix D between the two points, an N×N same-instance attention matrix A is constructed. same and an N×N non-identical instance attention matrix A diff ,in,

[0024]

[0025] A sameij Indicates A same The element in the i-th row and j-th column of the matrix, A diffij Indicates A diff The element in the i-th row and j-th column of the matrix.

[0026] The present invention also proposes a device for solving the posture of disordered stacked parts with weak textures based on deep learning, comprising the following:

[0027] A point cloud file making module is used to scan parts in a plane to obtain scanned point clouds, and to segment and preprocess the scanned point clouds to obtain point cloud files of the front and back sides of several parts;

[0028] A training data generation module, used for pre-generating a homogeneous transformation matrix of a batch of parts relative to a world coordinate system when the parts are stacked, so as to generate training data by reading the point cloud file and transforming it to a corresponding stacking position based on the homogeneous transformation matrix;

[0029] The network model training module is used to train the improved FPCC network based on the training data and save the weight file of the trained network.

[0030] Specifically, the improved FPCC network adds a feature point prediction module on the basis of the original FPCC network, and adds an attention matrix to the instance segmentation loss calculation of the original FPCC network;

[0031] The feature point position determination module is used to obtain the scanned point cloud of the target part, perform point cloud preprocessing on the scanned point cloud of the target part and input it into the weight file for prediction, obtain the output center score, point cloud instance label and feature point coordinates, and perform Hough voting to obtain the feature point position of the target part;

[0032] The homogeneous transformation matrix calculation module is used to obtain the homogeneous transformation matrix corresponding to the target part through ICP point cloud registration of feature point positions and feature point templates.

[0033] The pose solving module is used to find the workpiece whose registration error is lower than the preset threshold and output its pose based on the homogeneous transformation matrix corresponding to the target part.

[0034] The beneficial effects of the present invention are:

[0035] The present invention proposes a method for solving the pose of disordered stacked parts with weak textures based on deep learning, and automatically generates annotation data for network training through computer technology, which greatly reduces the cost of manual annotation. In terms of network design, the effect of point cloud instance segmentation is improved by adding the calculation of the attention matrix, and the added feature point prediction module enables the network to have the ability to solve the pose of parts. In the inference stage, the accurate position of the feature points is solved by the Hough voting method, which optimizes the error introduced by instance segmentation to a certain extent and increases the accuracy of the prediction. At the same time, compared with the direct point cloud registration method, the feature point registration method not only ensures a certain degree of registration accuracy but also greatly reduces the time loss, and also avoids the difference in algorithm performance due to different sensor sampling distances. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The above and other features of the present disclosure will become more apparent by describing in detail the embodiments shown in the accompanying drawings. The same reference numerals in the accompanying drawings of the present disclosure represent the same or similar elements. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure. For those skilled in the art, other accompanying drawings can be obtained based on these accompanying drawings without creative work. In the accompanying drawings:

[0037] Figure 1 Shown is a schematic diagram of a part point cloud collected in one embodiment of the present invention;

[0038] Figure 2 FIG. 1 is a schematic diagram of part feature points collected in one embodiment of the present invention;

[0039] Figure 3 It is a schematic diagram showing a homogeneous transformation matrix of a part when collecting training data in one embodiment of the present invention;

[0040] Figure 4 The figure shows a schematic diagram of a part point cloud in which all blocked points are removed by Open3D in one embodiment of the present invention;

[0041] Figure 5 Shown is a structural principle diagram of the improved FPCC network in the present invention;

[0042] Figure 6 The figure shows a schematic diagram of a scanning point cloud of a target part in one embodiment of the present invention;

[0043] Figure 7 The figure shows a segmentation effect diagram after segmenting the scanned point cloud of the target part in one embodiment of the present invention;

[0044] Figure 8 The figure shows the final effect after the scanning point cloud of the target part is processed in one embodiment of the present invention;

[0045] Fig. 9 Shown is a flow chart of a method for solving the pose of disorderly stacked parts with weak textures based on deep learning of the present invention;

[0046] Fig.10 Shown is a structural block diagram of the device for solving the posture of disordered stacked parts with weak textures based on deep learning of the present invention. DETAILED DESCRIPTION

[0047] The following will be combined with the embodiments and drawings to clearly and completely describe the concept, specific structure and technical effects of the present invention, so as to fully understand the purpose, scheme and effect of the present invention. It should be noted that the embodiments in this application and the features in the embodiments can be combined with each other without conflict. The same reference numerals used throughout the drawings indicate the same or similar parts.

[0048] Example 1, reference Fig. 9 The present invention proposes a method for solving the pose of disordered stacked parts with weak textures based on deep learning, which includes the following:

[0049] Step 110, scanning the parts in the plane to obtain scanning point clouds, and performing segmentation and preprocessing operations on the scanning point clouds to obtain point cloud files of the front and back sides of several parts;

[0050] Step 120, pre-generate a homogeneous transformation matrix of a batch of parts relative to the world coordinate system when the parts are stacked, and generate training data in a manner of reading the point cloud file and transforming it to a corresponding stacking position based on the homogeneous transformation matrix;

[0051] Step 130: training an improved FPCC network based on the training data, and saving a weight file of the trained network.

[0052] Specifically, the improved FPCC network adds a feature point prediction module on the basis of the original FPCC network, and adds an attention matrix to the instance segmentation loss calculation of the original FPCC network;

[0053] Step 140, obtaining a scanned point cloud of the target part, performing point cloud preprocessing on the scanned point cloud of the target part and inputting the point cloud into the weight file for prediction, obtaining an output center score, a point cloud instance label and feature point coordinates, and performing Hough voting to obtain the feature point position of the target part;

[0054] Step 150: Perform ICP point cloud registration with the feature point position and the feature point template to obtain the homogeneous transformation matrix corresponding to the target part.

[0055] Step 160 : Based on the homogeneous transformation matrix corresponding to the target part, find the workpiece whose registration error is lower than a preset threshold and output its position and posture.

[0056] In this embodiment 1, the training data of the deep learning network is automatically generated through the CAD model of the workpiece to solve the problem of high cost of training data annotation. At the same time, a network structure is proposed that can effectively solve the pose problem of disordered stacked parts with weak textures.

[0057] As a preferred embodiment of the present invention, specifically, scanning a part in a plane to obtain a scanning point cloud, segmenting and preprocessing the scanning point cloud to obtain point cloud files of the front and back sides of several parts, including:

[0058] The scanning point cloud of the part in the plane is collected by a 3D camera, and the point cloud belonging to the part is segmented from the scanning point cloud to obtain the part point cloud. A total of K part point clouds are collected from the front and back sides. At the same time, the CAD model of the part is collected, and the CAD model is sampled into a point cloud. The FPS farthest point sampling algorithm is used to sample K part feature points, and K point cloud files are constructed based on the part point cloud and part feature points.

[0059] In this preferred embodiment, the part is placed on a plane, and the point cloud of the part is collected by a 3D camera to segment the point cloud belonging to the part. A total of 10 part point clouds are collected from both the front and back sides. Figure 1 Read the CAD model of the object, sample the CAD model into a point cloud, and use the FPS farthest point sampling algorithm to sample K (10) part feature points, as shown in Figure 2 shown.

[0060] As a preferred embodiment of the present invention, specifically, pre-generating a homogeneous transformation matrix of parts relative to the world coordinate system when a batch of parts are stacked, including:

[0061] A rectangular material frame is placed in the simulation environment, and a batch of parts with random heights and random positions are generated within the range above the material frame. A batch of the parts are made to fall freely into the material frame. The number of parts is gradually increased from 3 to 50, and repeated 20 times. Figure 3 As shown in Figure 2. Record the homogeneous transformation matrix of each part relative to the world coordinate system when it is stationary.

[0062] As a preferred embodiment of the present invention, specifically, the process of generating training data includes:

[0063] Read the homogeneous transformation matrix of the saved parts relative to the world coordinate system one by one, randomly select the part point cloud and its corresponding part feature points in the collected point cloud file and transform them to the physical simulation position to obtain the transformed part point cloud and part feature points, and remove all the obscured points in the transformed part point cloud and part feature points, such as Figure 4 As shown, assuming that the single part point cloud after removing the occluded points has I points, let the i-th point of the workpiece point cloud be p i (x i ,y i ,z i ), i≤N, x i ,y i ,z i are the coordinate values ​​of the i-th point in the point cloud, and the center point of the workpiece point cloud is c, p iThe Euclidean distance from the point to the center point c is d i , calculate p respectively i The center score s i , point to feature point p k OffsetOf ki ,

[0064]

[0065] Of Ki =p i -p k

[0066] All points in the scene are written one by one as training data according to the point coordinates (3 channels), the normalized coordinates (3 channels), the instance labels (1 channel), the center scores (1 channel), and the feature point offsets (K×3 channels).

[0067] As a preferred embodiment of the present invention, a specific, improved FPCC network includes:

[0068] Designed as Figure 5 The network structure shown in the figure adds a feature point prediction module on the basis of the FPCC network, so that each point can estimate the coordinates of K feature points and then obtain the redundant feature point coordinates. After obtaining the redundant feature point coordinates, the Hough voting method is used to obtain the position of the feature point with the highest number of votes. Finally, the ICP point cloud registration method of the feature points is used to solve the accurate pose of the workpiece. At the same time, the attention matrix is ​​added to the instance segmentation loss calculation of the FPCC network, and an N×N same-instance attention matrix A is constructed according to the distance matrix D between the two points. same and an N×N non-identical instance attention matrix A diff ,in,

[0069]

[0070]

[0071] A sameij Indicates A same The element in the i-th row and j-th column of the matrix, A diffij Indicates A diff The element in the i-th row and j-th column of the matrix.

[0072] In this preferred embodiment, by constructing the network structure in the above manner, the distance between the features of the same instance point and its center point in the feature space can be made smaller; the distance between the features of points that are close in distance but belong to different instances in the feature space is farther, which effectively improves the instance segmentation effect of the network.

[0073] For the subsequent acquisition of the scanned point cloud of the target part, the scanned point cloud of the target part is preprocessed and then input into the weight file for prediction, and the output center score, point cloud instance label and feature point coordinates are obtained, and Hough voting is performed to obtain the feature point position of the target part;

[0074] The homogeneous transformation matrix corresponding to the target part is obtained by performing ICP point cloud registration with the feature point position and the feature point template.

[0075] The process of finding the workpiece whose registration error is lower than the preset threshold and outputting its position and posture based on the homogeneous transformation matrix corresponding to the target part.

[0076] That is, the inference stage, the input is Figure 6 The point cloud data shown in the figure obtains the center point score of each point, the feature vector of each point, and the estimated feature point coordinates through network inference. Select point i with the highest center point score, and then remove the points within d (d is the maximum radius of the workpiece) around point i from the candidate point set. Repeat the selection of the point with the highest score in the candidate point set until the candidate point set is empty. Calculate the distance between the selected feature point and the features of all other points in the feature space, and select the point within a distance of 1 as the instance point cloud of the center point. The point cloud instance segmentation effect is shown in the figure below. Figure 7 As shown. Traverse the points belonging to the same instance, obtain the predicted feature point position of each point, and use the knn algorithm to solve the point with the most neighbors for a single feature point as the accurate position of the feature point. Finally, perform point cloud registration with the feature point template to complete the pose estimation of the part. Select the workpiece with higher confidence through the root mean square error of the point cloud, transform the CAD model to the output homogeneous transformation matrix position to make the model point black. The final effect is as follows: Figure 8 shown.

[0077] Reference Fig.10 The present invention also proposes a device for solving the pose of disordered stacked parts with weak textures based on deep learning, comprising the following:

[0078] The point cloud file making module 100 is used to scan the parts in the plane to obtain the scanned point cloud, and to segment and pre-process the scanned point cloud to obtain the point cloud files of the front and back sides of several parts;

[0079] The training data generation module 200 is used to pre-generate a homogeneous transformation matrix of a batch of parts relative to a world coordinate system when the parts are stacked, so as to generate training data by reading the point cloud file and transforming it to a corresponding stacking position based on the homogeneous transformation matrix;

[0080] The network model training module 300 is used to train the improved FPCC network based on the training data and save the weight file of the trained network.

[0081] Specifically, the improved FPCC network adds a feature point prediction module on the basis of the original FPCC network, and adds an attention matrix to the instance segmentation loss calculation of the original FPCC network;

[0082] The feature point position determination module 400 is used to obtain the scanned point cloud of the target part, perform point cloud preprocessing on the scanned point cloud of the target part and input it into the weight file for prediction, obtain the output center score, point cloud instance label and feature point coordinates, and perform Hough voting to obtain the feature point position of the target part;

[0083] The homogeneous transformation matrix calculation module 500 is used to obtain the homogeneous transformation matrix corresponding to the target part by performing ICP point cloud registration with the feature point position and the feature point template.

[0084] The posture solving module 600 is used to find the workpiece whose registration error is lower than a preset threshold and output its posture based on the homogeneous transformation matrix corresponding to the target part.

[0085] In addition, each functional module in each embodiment of the present invention may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module. The above integrated modules may be implemented in the form of hardware or in the form of software functional modules.

[0086] If the integrated module is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or system that can carry the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal and software distribution medium, etc.

[0087] Although the description of the present invention has been quite detailed and specifically describes several described embodiments, it is not intended to be limited to any of these details or embodiments or any particular embodiment, but should be regarded as providing a broad possible interpretation of these claims in view of the prior art by reference to the appended claims, thereby effectively covering the intended scope of the present invention. In addition, the above description of the present invention is based on the embodiments foreseeable by the inventor, and its purpose is to provide a useful description, and those non-substantial changes to the present invention that have not yet been foreseen may still represent equivalent changes to the present invention.

[0088] The above is only a preferred embodiment of the present invention. The present invention is not limited to the above implementation. As long as the technical effect of the present invention is achieved by the same means, it should belong to the protection scope of the present invention. Within the protection scope of the present invention, its technical scheme and / or implementation method can have various modifications and changes.

Claims

1. A pose solving method for disorderly stacked parts with weak textures based on deep learning, characterized in that: Includes the following: Scanning the parts in the plane to obtain scanning point clouds, and performing segmentation and preprocessing operations on the scanning point clouds to obtain point cloud files of the front and back sides of several parts; Pre-generating a homogeneous transformation matrix of a batch of parts relative to a world coordinate system when the parts are stacked, so as to generate training data in a manner of reading the point cloud file and transforming it to a corresponding stacking position based on the homogeneous transformation matrix; Based on the training data, the improved FPCC network is trained and the weight file of the trained network is saved. Specifically, the improved FPCC network adds a feature point prediction module on the basis of the original FPCC network, and adds an attention matrix to the instance segmentation loss calculation of the original FPCC network; Obtaining a scanned point cloud of the target part, performing point cloud preprocessing on the scanned point cloud of the target part and inputting the point cloud into the weight file for prediction, obtaining an output center score, a point cloud instance label and feature point coordinates, and performing Hough voting to obtain the feature point position of the target part; The homogeneous transformation matrix corresponding to the target part is obtained by performing ICP point cloud registration with the feature point position and the feature point template. Based on the homogeneous transformation matrix corresponding to the target part, the workpiece with a registration error lower than a preset threshold is found and its position and posture are output.

2. The method for solving the pose of disordered stacked parts with weak textures based on deep learning according to claim 1 is characterized in that: Specifically, a part in a scanning plane is scanned to obtain a scanning point cloud, and the scanning point cloud is segmented and preprocessed to obtain point cloud files of the front and back sides of several parts, including: The scanning point cloud of the part in the plane is collected by a 3D camera, and the point cloud belonging to the part is segmented from the scanning point cloud to obtain the part point cloud. A total of X part point clouds are collected from both the front and back sides. At the same time, the CAD model of the part is collected, and the CAD model is sampled into a point cloud. K part feature points are sampled using the FPS farthest point sampling algorithm, and X point cloud files are constructed based on the part point cloud and the part feature points.

3. The method for solving the pose of disorderly stacked parts with weak textures based on deep learning according to claim 2, characterized in that: Specifically, a homogeneous transformation matrix of a batch of parts relative to the world coordinate system is pre-generated when the parts are stacked, including: A rectangular material frame is placed in the simulation environment, and a batch of parts with random heights and random positions are generated above the material frame. The batch of parts are made to fall freely into the material frame. The number of parts gradually increases from 3 to 50, and this is repeated 20 times. The homogeneous transformation matrix of each part relative to the world coordinate system when it is stationary is recorded.

4. The method for solving the position and posture of disordered stacked parts with weak textures based on deep learning according to claim 3 is characterized in that: Specifically, the process of generating training data includes: Read the homogeneous transformation matrix of the saved parts relative to the world coordinate system one by one, randomly select the part point cloud and its corresponding part feature points in the collected point cloud file, and transform them to the physical simulation position to obtain the transformed part point cloud and part feature points Points, remove all the obscured points in the transformed part point cloud and part feature points, assume that a single part point cloud has N points, let the i-th point of the workpiece point cloud be p i (x i ,y i ,z i ), i≤N, x i ,y i ,z i are the coordinate values ​​of the i-th point in the point cloud, and the center point of the workpiece point cloud is c, p i The Euclidean distance from the point to the center point c is d i , calculate p respectively i The center score s i , point to feature point p k OffsetOf ki , Of Ki =p i -p k All points in the scene are used as training data according to their coordinates, normalized coordinates, instance labels, center scores, and feature point offsets.

5. The method for solving the pose of disorderly stacked parts with weak textures based on deep learning according to claim 1, characterized in that: Specifically, the improved FPCC network includes: A feature point prediction module is added on the basis of the FPCC network, so that each point can estimate the coordinates of K feature points and then obtain the redundant feature point coordinates. After obtaining the redundant feature point coordinates, the Hough voting method is used to obtain the position of the feature point with the highest number of votes. Finally, the ICP point cloud registration method of the feature points is used to solve the accurate pose of the workpiece. At the same time, the attention matrix is ​​added to the instance segmentation loss calculation of the FPCC network. According to the distance matrix D between two points, an N×N same-instance attention matrix A is constructed. same and an N×N non-identical instance attention matrix A diff ,in, A sameij Indicates A same The element in the i-th row and j-th column of the matrix, A diffij Indicates A diff The element in the i-th row and j-th column of the matrix.

6. A device for solving the pose of disordered stacked parts with weak textures based on deep learning, characterized in that: Includes the following: A point cloud file making module is used to scan parts in a plane to obtain scanned point clouds, and to segment and preprocess the scanned point clouds to obtain point cloud files of the front and back sides of several parts; A training data generation module, used for pre-generating a homogeneous transformation matrix of a batch of parts relative to a world coordinate system when the parts are stacked, so as to generate training data by reading the point cloud file and transforming it to a corresponding stacking position based on the homogeneous transformation matrix; The network model training module is used to train the improved FPCC network based on the training data and save the weight file of the trained network. Specifically, the improved FPCC network adds a feature point prediction module on the basis of the original FPCC network, and adds an attention matrix to the instance segmentation loss calculation of the original FPCC network; The feature point position determination module is used to obtain the scanned point cloud of the target part, perform point cloud preprocessing on the scanned point cloud of the target part and input it into the weight file for prediction, obtain the output center score, point cloud instance label and feature point coordinates, and perform Hough voting to obtain the feature point position of the target part; The homogeneous transformation matrix calculation module is used to obtain the homogeneous transformation matrix corresponding to the target part through ICP point cloud registration of feature point positions and feature point templates. The pose solving module is used to find the workpiece whose registration error is lower than the preset threshold and output its pose based on the homogeneous transformation matrix corresponding to the target part.

Images