Common shoe last gluing track correction method based on depth camera

Abstract

The present application relates to a kind of common shoe last glue coating track correction method based on depth camera, builds glue coating track correction system, obtains the conversion matrix of depth camera to six-axis robot coordinate system, and common shoe last is obtained as template shoe by being put on shoe bag, template shoe point cloud is obtained, and glue coating track based on template shoe is obtained, is photographed by binocular depth camera and is located in control fixture tooling on the shoe to be coated glue and obtains the point cloud of shoe to be coated glue, respectively, the demarcation of template shoe point cloud and the point cloud of shoe to be coated glue, based on the demarcation after each point cloud obtains first conversion matrix and second conversion matrix, the demarcation processing of all points of glue coating track based on mean point abscissa obtains first point set and second point set, respectively, the conversion of first conversion matrix to first point set and the conversion of second conversion matrix to second point set are carried out, the track formed by first conversion point set and second conversion point set together is used as the glue coating track of common shoe last after correction, so that common shoe last has obtained high-precision glue coating track.

Description

Technical Field

[0001] This invention relates to the field of shoemaking, and more particularly to a method for correcting the adhesive application trajectory of ordinary shoe lasts based on a depth camera. Background Technology

[0002] A shoe last, a template used in shoe making, is mainly used to control the shape and structure of shoes. In the past, shoe lasts played an important role in handmade production.

[0003] With the development of automation technology, automation in shoe manufacturing has become a common practice. Current automated processes in shoe production include packaging, polishing, and gluing. These processes also require shoe lasts to control the size and shape of the shoes, ensuring that each shoe is very close in size and shape, thus guaranteeing quality. Using shoe lasts reduces worker time and costs, increases production efficiency, and minimizes potential errors.

[0004] Most existing automated gluing processes utilize robotic arms to replace manual production. The main process of automated gluing involves first teaching a template shoe and recording its trajectory, then using a robotic arm to repeat this recorded trajectory and perform the gluing operation on all shoes of the same style as the template shoe. Because shoes are made of flexible materials and cannot be fixed in shape, it is necessary to use high-precision standard shoe lasts in production, and to control the precision of the standard shoe lasts to within the millimeter level.

[0005] However, existing methods for processing the adhesive application trajectory of ordinary shoe lasts have shortcomings: as customers' personalized needs continue to emerge, automated shoe manufacturing often requires the customization of various types of high-precision standard shoe lasts in small batches. However, existing ordinary shoe lasts cannot meet the requirements of high-precision standard shoe lasts, and therefore cannot meet the needs of automated shoe manufacturing based on high-precision standard shoe lasts. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to provide a method for correcting the adhesive application trajectory of ordinary shoe lasts based on a depth camera, which is an improvement over the above-mentioned prior art.

[0007] The technical solution adopted by the present invention to solve the above-mentioned technical problems is: a method for correcting the adhesive application trajectory of ordinary shoe lasts based on a depth camera, characterized by comprising the following steps:

[0008] Step 1: Build a glue application trajectory correction system based on a depth camera and a six-axis robot; the glue application trajectory correction system includes a fixture for placing shoe lasts, a binocular depth camera, and a six-axis robot equipped with a glue application head; the binocular depth camera is responsible for collecting the three-dimensional data of ordinary shoe lasts located on the fixture.

[0009] Step 2: Process the depth camera coordinate system and the six-axis robot coordinate system to obtain the transformation matrix from the depth camera coordinate system to the six-axis robot coordinate system;

[0010] Step 3: Put a shoe bag that meets the preset requirements into a regular shoe last as a template shoe, and fit the template shoe tightly to the sole. Then use a binocular depth camera to take pictures of the shoe to be coated with glue to obtain the point cloud of the template shoe.

[0011] Step 4: Outline the sole on the shoe bag to obtain the glue application trajectory based on the template shoe;

[0012] Step 5: After removing the template shoe from the fixture, place the shoe to be coated onto the fixture and use a binocular depth camera to photograph the shoe to be coated to obtain the point cloud of the shoe.

[0013] Step 6: Calculate the average of the maximum and minimum x-coordinate values ​​in the template shoe point cloud, and use this average as the first point cloud boundary value; classify all points in the template shoe point cloud that are greater than the first point cloud boundary value as the first point cloud, and classify all remaining points in the template shoe point cloud as the second point cloud;

[0014] Step 7: Calculate the average of the maximum and minimum abscissa values ​​in the point cloud of the shoe to be coated, and use this average value as the second point cloud boundary value; divide all points in the point cloud of the shoe to be coated that are greater than the second point cloud boundary value into the third point cloud, and divide all remaining points in the point cloud of the shoe to be coated into the fourth point cloud;

[0015] Step 8: Perform point-to-surface ICP registration on the first point cloud and the third point cloud to obtain the first transformation matrix; and perform point-to-surface ICP registration on the second point cloud and the fourth point cloud to obtain the second transformation matrix.

[0016] Step 9: Obtain the mean coordinates of all trajectory points on the obtained adhesive application trajectory, and take the trajectory points located on the adhesive application trajectory that have a x-coordinate greater than the mean coordinate as the first point set, and take the trajectory points located on the adhesive application trajectory that have a x-coordinate less than the mean coordinate as the second point set; wherein, the mean coordinates include the three-dimensional coordinate values ​​of the mean point;

[0017] Step 10: Transform the first point set using the first transformation matrix to obtain the first transformed point set, and transform the second point set using the second transformation matrix to obtain the second transformed point set. The trajectory formed by the first transformed point set and the second transformed point set together is used as the corrected ordinary shoe last gluing trajectory.

[0018] Improvedly, in the conventional shoe last gluing trajectory correction method based on a depth camera, the binocular depth camera is positioned from top to bottom facing the bottom of the shoe last of the fixture, and the binocular depth camera is at a 45° angle to the horizontal ground; the camera parameters of the binocular depth camera are: 2.3*10^6 pixels, camera resolution of 1920*1200, and spatial resolution of 0.24-1.07mm; the six-axis robot is an ABB six-axis robot.

[0019] In a further improvement, in the conventional shoe last adhesive application trajectory correction method based on a depth camera, step 2, the transformation matrix processing from the depth camera's internal coordinates to the six-axis robot coordinate system, includes the following steps:

[0020] Step a1: Replace the glue-applying head on the six-axis robot with a calibration tip, and use the calibration tip to sequentially touch each column on the pre-set calibration plate to obtain the three-dimensional coordinates corresponding to each column. All obtained three-dimensional coordinates form a first three-dimensional coordinate set. The pre-set calibration plate has nine cuboid columns, each with a cross-shaped groove. The first three-dimensional coordinate set is labeled S. ROBOT9 ;

[0021] Step a2: Use a binocular depth camera to photograph the pre-set calibration board, obtain the point cloud of the pre-set calibration board, and then apply grayscale information to the point cloud. Next, use the 3D point cloud processing software Cloudcompare to process the grayscale-added point cloud to obtain the 3D coordinates corresponding to each pillar. All obtained 3D coordinates form a second 3D coordinate set; the second 3D coordinate set is denoted as S. CAMERA9 ;

[0022] Step a3: Perform matrix transformation on the first three-dimensional coordinate set to obtain a first three-dimensional matrix; and perform matrix transformation on the second three-dimensional coordinate set to obtain a second three-dimensional matrix; wherein, the first three-dimensional matrix is ​​denoted as G. ROBOT9 The second three-dimensional matrix is ​​labeled G. CAMERA9 ;

[0023] Step a4: Based on the obtained first three-dimensional matrix and second three-dimensional matrix, process to obtain the transformation matrix; wherein, the transformation matrix is ​​labeled as H, G CAMERA9 H=G ROBOT9 .

[0024] Furthermore, in the aforementioned method for correcting the trajectory of adhesive application on ordinary shoe lasts based on a depth camera, the second three-dimensional coordinate set S CAMERA9 Nine points are labeled {c1, c2...c9}, and nine points in the first three-dimensional coordinate set are labeled {r1, r2...r9}, with each point having coordinates in the form {xyz}.T The transformation matrix H is as follows:

[0025]

[0026] Among them, R 3×3 T is a 3×3 rotation matrix. 3×1 It is a 3×1 translation matrix; α represents the rotation angle around the X-axis, β represents the rotation angle around the Y-axis, γ represents the rotation angle around the Z-axis, t1 is the displacement distance on the X-axis, t2 is the displacement distance on the Y-axis, and t3 is the displacement distance on the Z-axis.

[0027] In a further improvement, the transformation matrix H in the conventional shoe last adhesive trajectory correction method based on a depth camera is calculated as follows:

[0028] Step b1, calculate the optimal translation matrix; where:

[0029] Let the equation Taking the derivative, we get:

[0030]

[0031] make have to

[0032] Step b2, calculate the optimal rotation matrix; where:

[0033] Without considering translation, let the equations be...

[0034] ||R 3×3 c i -r i || 2 =(R 3×3 c i -r i ) T (R 3×3 c i -r i )=||c i || 2 +||r i || 2 -2c i T R 3×3 r i ;

[0035] Among them, ||c i || 2 +||r i || 2 To determine the value, calculate The minimum value;

[0036] Change the original equation to find The maximum value, C = [c1, c1...c9], R robot = [r1, r1... r9];

[0037] trace(C T R 3×3 R robot =trace(R) 3×3 R robot C T );

[0038] R robot C T Perform SVD decomposition to obtain trace(R) 3×3 R robot C T ) = trace(ΣV T R 3×3 U), where V,R 3×3 U are both orthogonal matrices; let V T R 3×3 U is the identity matrix, and the identity matrix V is... T R 3×3 U corresponds to If the value is the largest, then R 3×3 =VU T ; where the matrix R 3×3 Optimal rotation matrix;

[0039] Step b3: Merge the obtained optimal rotation matrix into a 4*4 matrix; wherein, the 4*4 matrix obtained after fusion is the transformation matrix H.

[0040] Compared with the prior art, the advantages of this invention are as follows: The ordinary shoe last gluing trajectory correction method of this invention establishes a gluing trajectory correction system based on a depth camera and a six-axis robot, obtains the transformation matrix from the coordinate system of the depth camera to the coordinate system of the six-axis robot, puts an ordinary shoe last on a shoe bag that meets the preset requirements as a template shoe, obtains the point cloud of the template shoe, and performs outlining processing on the sole of the shoe bag to obtain the gluing trajectory based on the template shoe. A binocular depth camera is used to photograph the shoe to be glued located on the control fixture to obtain the point cloud of the shoe to be glued. The point cloud of the template shoe and the shoe to be glued are then compared. The point cloud of the glued shoe is divided into boundaries, and a first transformation matrix and a second transformation matrix are obtained based on the point cloud after the boundary. Then, all points of the glued trajectory are divided into boundaries based on the x-coordinate of the mean point to obtain a first point set and a second point set. The first point set is transformed using the first transformation matrix to obtain a first transformed point set, and the second point set is transformed using the second transformation matrix to obtain a second transformed point set. The trajectory formed by the first transformed point set and the second transformed point set together is used as the corrected glued trajectory of the ordinary shoe last. Thus, a high-precision glued trajectory is obtained using an ordinary shoe last. Attached Figure Description

[0041] Figure 1 This is a schematic diagram of the process for correcting the adhesive trajectory of a conventional shoe last based on a depth camera, as described in an embodiment of the present invention. Detailed Implementation

[0042] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0043] This embodiment provides a method for correcting the adhesive application trajectory of ordinary shoe lasts based on a depth camera. Specifically, see [link to documentation]. Figure 1 As shown, the method for correcting the adhesive application trajectory of a conventional shoe last based on a depth camera in this embodiment includes the following steps:

[0044] Step 1: Construct a glue application trajectory correction system based on a depth camera and a six-axis robot. This system includes a fixture for placing shoe lasts, a binocular depth camera, and a six-axis robot equipped with a glue application head. The binocular depth camera collects 3D data of ordinary shoe lasts located on the fixture. The camera is positioned from top to bottom, facing the bottom of the shoe last on the fixture, at a 45° angle to the horizontal ground. The camera parameters are: 2.3*10^6 pixels, 1920*1200 resolution, and 0.24-1.07mm spatial resolution. The six-axis robot is an ABB six-axis robot.

[0045] Step 2: Process the depth camera coordinate system and the six-axis robot coordinate system to obtain the transformation matrix from the depth camera coordinate system to the six-axis robot coordinate system;

[0046] Step 3: Put a shoe bag that meets the preset requirements into a regular shoe last as a template shoe, and fit the template shoe tightly to the sole. Then use a binocular depth camera to take pictures of the shoe to be coated with glue to obtain the point cloud of the template shoe.

[0047] Step 4: Outline the sole on the shoe bag to obtain the glue application trajectory based on the template shoe;

[0048] Step 5: After removing the template shoe from the fixture, place the shoe to be coated onto the fixture and use a binocular depth camera to photograph the shoe to be coated to obtain the point cloud of the shoe.

[0049] Step 6: Calculate the average of the maximum and minimum x-coordinate values ​​in the template shoe point cloud, and use this average as the first point cloud boundary value; classify all points in the template shoe point cloud that are greater than the first point cloud boundary value as the first point cloud, and classify all remaining points in the template shoe point cloud as the second point cloud;

[0050] Step 7: Calculate the average of the maximum and minimum abscissa values ​​in the point cloud of the shoe to be coated, and use this average value as the second point cloud boundary value; divide all points in the point cloud of the shoe to be coated that are greater than the second point cloud boundary value into the third point cloud, and divide all remaining points in the point cloud of the shoe to be coated into the fourth point cloud;

[0051] Step 8: Perform point-to-surface ICP registration on the first point cloud and the third point cloud to obtain the first transformation matrix; and perform point-to-surface ICP registration on the second point cloud and the fourth point cloud to obtain the second transformation matrix; wherein, point-to-surface ICP registration is a mature existing technology and will not be described in detail here.

[0052] Step 9: Obtain the mean coordinates of all trajectory points on the obtained adhesive application trajectory, and take the trajectory points located on the adhesive application trajectory that have a x-coordinate greater than the mean coordinate as the first point set, and take the trajectory points located on the adhesive application trajectory that have a x-coordinate less than the mean coordinate as the second point set; wherein, the mean coordinates include the three-dimensional coordinate values ​​of the mean point;

[0053] Step 10: Transform the first point set using the first transformation matrix to obtain the first transformed point set, and transform the second point set using the second transformation matrix to obtain the second transformed point set. The trajectory formed by the first transformed point set and the second transformed point set together is used as the corrected ordinary shoe last gluing trajectory.

[0054] Specifically, in this embodiment, in step 2 above, the transformation matrix processing from the depth camera's internal coordinates to the six-axis robot coordinate system includes the following steps a1 to a4:

[0055] Step a1: Replace the dispensing head on the six-axis robot with a calibration tip, and use the calibration tip to sequentially touch each column on the pre-set calibration plate to obtain the three-dimensional coordinates corresponding to each column. All obtained three-dimensional coordinates form a first three-dimensional coordinate set. The pre-set calibration plate has 9 cuboid columns, each with a cross-shaped groove. The first three-dimensional coordinate set is labeled S. ROBOT9 ;

[0056] Step a2: Use a binocular depth camera to photograph the pre-set calibration board, obtain the point cloud of the pre-set calibration board, and then apply grayscale information to the point cloud. Next, use the 3D point cloud processing software Cloudcompare to process the grayscale-added point cloud to obtain the 3D coordinates corresponding to each pillar. All obtained 3D coordinates form a second 3D coordinate set; the second 3D coordinate set is denoted as S. CAMERA9 ;

[0057] Step a3: Perform matrix transformation on the first three-dimensional coordinate set to obtain a first three-dimensional matrix; and perform matrix transformation on the second three-dimensional coordinate set to obtain a second three-dimensional matrix; wherein, the first three-dimensional matrix is ​​denoted as G. ROBOT9 The second three-dimensional matrix is ​​labeled G. CAMERA9 ;

[0058] Step a4: Based on the obtained first three-dimensional matrix and second three-dimensional matrix, process to obtain the transformation matrix; wherein, the transformation matrix is ​​labeled as H, G CAMERA9 H=G ROBOT9 .

[0059] In this embodiment, the second three-dimensional coordinate set S CAMERA9 Nine points are labeled {c1, c2...c9}, and nine points in the first three-dimensional coordinate set are labeled {r1, r2...r9}, with coordinates in the form {xyz}. T The transformation matrix H is as follows:

[0060]

[0061] Among them, R 3×3 T is a 3×3 rotation matrix. 3×1 It is a 3×1 translation matrix; α represents the rotation angle around the X-axis, β represents the rotation angle around the Y-axis, γ represents the rotation angle around the Z-axis, t1 is the displacement distance on the X-axis, t2 is the displacement distance on the Y-axis, and t3 is the displacement distance on the Z-axis.

[0062] Specifically, the transformation matrix H involved in this embodiment is calculated as follows:

[0063] Step b1, calculate the optimal translation matrix; where:

[0064] Let the equation Taking the derivative, we get:

[0065]

[0066] make have to

[0067] Step b2, calculate the optimal rotation matrix; where:

[0068] Without considering translation, let the equations be...

[0069] ||R 3×3 c i -r i || 2 =(R 3×3 c i -r i ) T (R 3×3 c i -r i )=||c i || 2 +||r i || 2 -2c i T R 3×3 r i ;

[0070] Among them, ||c i || 2 +||r i || 2 To determine the value, calculate The minimum value;

[0071] Change the original equation to find The maximum value, C = [c1, c1...c9], R robot = [r1, r1... r9];

[0072] trace(C T R 3×3 R robot =trace(R) 3×3 R robot C T );

[0073] R robot C T Perform SVD decomposition to obtain trace(R) 3×3 R robotC T ) = trace(ΣV T R 3×3 U), where V,R 3×3 U are both orthogonal matrices; let V T R 3×3 U is the identity matrix, and the identity matrix V is... T R 3×3 U corresponds to If the value is the largest, then R 3×3 =VU T ; where the matrix R 3×3 Optimal rotation matrix;

[0074] Step b3: Merge the obtained optimal rotation matrix into a 4*4 matrix; wherein, the 4*4 matrix obtained after fusion is the transformation matrix H.

[0075] Although preferred embodiments of the present invention have been described in detail above, it should be clearly understood that various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A common shoe last glue coating track correction method based on a depth camera, characterized in that, Comprising the following steps: Step 1, build a glue track correction system based on a depth camera and a six-axis robot; wherein the glue track correction system comprises a clamp tool for placing a shoe tree, a binocular depth camera and a six-axis robot provided with a glue head; the binocular depth camera is responsible for collecting the three-dimensional data of the ordinary shoe tree located on the clamp tool; Step 2, process the depth camera coordinate system and the six-axis robot coordinate system to obtain the conversion matrix from the depth camera internal coordinate to the six-axis robot coordinate system; Step 3, put the shoe cover that meets the preset requirements on the ordinary shoe tree as a template shoe, and tightly fit the template shoe with the sole, and then use the binocular depth camera to shoot the shoe to be glued to obtain the point cloud of the template shoe; Step 4, perform edge processing on the sole on the shoe cover to obtain the glue track based on the template shoe; Step 5, after removing the template shoe located on the clamp tool, place the shoe to be glued on the clamp tool, and use the binocular depth camera to shoot the shoe to be glued to obtain the point cloud of the shoe to be glued; Step 6, calculate the average value of the maximum horizontal coordinate value and the minimum horizontal coordinate value in the template shoe point cloud, and take the average value as the first point cloud division value; all points located in the template shoe point cloud and greater than the first point cloud division value are divided into a first point cloud, and all remaining points located in the template shoe point cloud are divided into a second point cloud; Step 7, calculate the average value of the maximum horizontal coordinate value and the minimum horizontal coordinate value in the template shoe point cloud, and take the average value as the second point cloud division value; all points located in the template shoe point cloud and greater than the second point cloud division value are divided into a third point cloud, and all remaining points located in the template shoe point cloud are divided into a fourth point cloud; Step 8, perform ICP registration processing based on point-to-plane on the first point cloud and the third point cloud to obtain a first conversion matrix; and perform ICP registration processing based on point-to-plane on the second point cloud and the fourth point cloud to obtain a second conversion matrix; Step 9, obtain the average point coordinate of all track points on the obtained glue track, and take the track points located on the glue track and having a greater average point horizontal coordinate as a first point set, and take the track points located on the glue track and having a smaller average point horizontal coordinate as a second point set; wherein the average point coordinate includes the three-dimensional coordinate value of the average point; Step 10, convert the first point set using the first conversion matrix to obtain a first converted point set, and convert the second point set using the second conversion matrix to obtain a second converted point set, and the track formed by the first converted point set and the second converted point set together is the corrected ordinary shoe tree glue track; In step 2, the conversion matrix from the depth camera internal coordinate to the six-axis robot coordinate system processing process comprises the following steps: Step a1, replace the glue head on the six-axis robot with a calibration tip, and use the calibration tip to sequentially touch each column on the pre-set calibration plate to obtain a three-dimensional coordinate corresponding to each column, and form a first three-dimensional coordinate set from all the obtained three-dimensional coordinates; wherein the pre-set calibration plate has nine cuboid columns, each of which has a cross-shaped groove; the first three-dimensional coordinate set is marked as S ROBOT9 ; Step a2, using binocular depth camera to shoot the preset calibration board, obtaining the point cloud of the preset calibration board, and then using three-dimensional point cloud processing software Cloudcompare to process the point cloud with gray information, respectively obtaining three-dimensional coordinates corresponding to each column, and forming a second three-dimensional coordinate set from all the obtained three-dimensional coordinates; wherein the second three-dimensional coordinate set is marked as S CAMERA9 ; Step a3, matrixing the first three-dimensional coordinate set to obtain a first three-dimensional matrix; and matrixing the second three-dimensional coordinate set to obtain a second three-dimensional matrix; wherein the first three-dimensional matrix is marked as G ROBOT9 , and the second three-dimensional matrix is marked as G CAMERA9 ; Step a4, based on the obtained first three-dimensional matrix and second three-dimensional matrix, processing to obtain the conversion matrix; wherein the conversion matrix is marked as H, G CAMERA9 H= G ROBOT9 .

2. The deep camera-based common last gluing trajectory correction method according to claim 1, wherein, The binocular depth camera is above the last bottom of the fixture tool, and the binocular depth camera is at an angle of 45° with the horizontal ground; the camera parameters of the binocular depth camera are as follows: the pixel number is 2.3*10 6 6, the camera resolution is 1920*1200, and the spatial resolution is 0.24-1.07 mm; the six-axis robot is an ABB six-axis robot.

3. The deep camera based common last gluing trajectory correction method of claim 1, wherein, The second three-dimensional coordinate set S CAMERA9 The nine points in the first three-dimensional coordinate set are respectively marked as The nine points in the first three-dimensional coordinate set are respectively marked as The coordinates of each point are in the form of The conversion matrix H is as follows: wherein, is a 3x3 rotation matrix, is a 3x1 translation matrix; denotes the rotation angle around the X-axis, denotes the rotation angle around the Y-axis, denotes the rotation angle around the Z-axis, t1 is the displacement distance in the X-axis, t2 is the displacement distance in the Y-axis, and t3 is the displacement distance in the Z-axis.

Images