3D vision-based shoe body glue spraying track extraction method

Abstract

The application discloses a shoe body glue spraying track extraction method based on 3D vision, comprising the following steps: using a ball center calibration method to calibrate a plurality of cameras, and obtaining a calibration matrix converted to a base coordinate system; obtaining splicing point clouds of a shoe body without a sole and a whole shoe with a sole in the base coordinate system respectively, and deleting interfering useless points; in the base coordinate system, accessing three-dimensional coordinates of corresponding points of the shoe body point cloud and the whole shoe point cloud, searching for an overlapping area to extract a glue line, and slicing the shoe body point cloud to find the lowest point as the glue line track; and adopting a curve fitting method to smooth the track as a glue spraying track of a robot glue spraying process. The method is used for automatic extraction of a glue line and glue spraying operation in shoe production, and can significantly improve production efficiency, reduce labor cost, and ensure glue spraying quality and product consistency.

Description

Technical Field

[0001] This invention pertains to image processing and point cloud visual computing, specifically to the automated extraction of adhesive line trajectories in footwear production, and more specifically to a method for extracting adhesive spraying trajectories from the shoe body based on 3D vision. Background Technology

[0002] In traditional gluing processes, the gluing process at the joint between the shoe body and the upper often results in missed areas and excess glue due to human error. This not only weakens the strong bond between the sole and the shoe body, directly affecting product quality, but also causes aesthetic problems due to uneven glue application. Excessive glue application not only causes excess glue to accumulate on the shoe edges, affecting the appearance, but more importantly, removing this excess glue often damages the shoe body, thus increasing the breakage rate and repair costs. Furthermore, the uncontrollable nature of manual operation, such as glue application deviations caused by hand tremors and the difficulty in ensuring uniformity, all become bottlenecks restricting production efficiency and quality improvement.

[0003] In addition, existing adhesive application trajectory extraction methods also have the following problems:

[0004] 1) Take point cloud images of the same shoe body at the same position from four different perspectives. Then, simply aggregate the point clouds taken by the four cameras together using a calibration matrix to create a complete shoe body point cloud. The point cloud carries a lot of background noise.

[0005] 2) In order to remove the redundant point cloud around the shoe body, the parameters need to be constantly changed when using the pass filter, and the point cloud of the shoe body cannot be directly extracted.

[0006] 3) However, the trajectory extracted using existing methods is not smooth and may have convex points around it.

[0007] Therefore, the existing methods for obtaining the adhesive trajectory described above cannot effectively extract the adhesive lines. Summary of the Invention

[0008] In view of the defects and shortcomings of the prior art described above, the purpose of this invention is to provide a 3D vision-based method for extracting the adhesive spray trajectory of a shoe body, thereby solving the problems existing in the prior art. In this method, the sphere center calibration method is used to calculate the transformation matrix between the four cameras; maximum Euclidean clustering is used to directly segment the complete shoe body point cloud without changing the parameters; and spline interpolation functions are used to fit the point cloud, making the fitted trajectory both smooth and close to the original point cloud data.

[0009] The technical solution adopted by the present invention to achieve the above objectives is as follows:

[0010] A 3D vision-based method for extracting adhesive spraying trajectories from shoe uppers involves the following steps:

[0011] S1. Use the sphere center calibration method to calibrate multiple cameras and obtain the calibration matrix for transformation to the base coordinate system;

[0012] S2. For the bottomless shoe body and the whole shoe with the sole, obtain the spliced ​​point cloud of the two in the base coordinate system, and delete the interfering and useless points;

[0013] S3. Under the base coordinate system, access the three-dimensional coordinates of the corresponding points of the matching shoe body point cloud and the whole shoe point cloud, search the overlapping area to extract the glue line, and slice the shoe body point cloud to find the lowest point as the glue line trajectory.

[0014] S4. The trajectory is smoothed using curve fitting method and used as the glue spraying trajectory for the robot glue spraying process.

[0015] The ball center calibration method includes:

[0016] The calibration sphere is placed under the view of multiple cameras, triggering multiple cameras to simultaneously capture multiple sets of point clouds of the calibration sphere at different positions;

[0017] The point cloud of the calibration sphere is filtered and then fitted to a 3D point cloud sphere.

[0018] One of the multiple cameras is selected as the base coordinate system, and the other cameras are calibrated on the base coordinate system. The calibration sphere point cloud is then fitted to obtain the complete calibration sphere point cloud.

[0019] The filtration process includes:

[0020] Extract the spherical point cloud from the cluttered point cloud;

[0021] Use a pass-through filter to set a threshold to remove cluttered point clouds around the sphere;

[0022] The RANSAC plane fitting principle is used to remove the planar point cloud below the sphere.

[0023] The step of calibrating other cameras on the base coordinate system includes:

[0024] Calculate the center of the 3D point cloud sphere, and use SVD to calculate the calibration matrix between any camera outside the base coordinate system and the base coordinate system;

[0025] The images of the sphere captured by the four cameras are fitted to the base coordinate system using the calibration matrix described above to obtain a complete sphere, thus completing the calibration.

[0026] Step S2 includes:

[0027] Multiple cameras are used to scan the shoe body and the whole shoe from different angles to obtain point cloud data of the two in different poses. The calibration matrix is ​​used to transform the data to the base coordinate system and the point clouds of the two are stitched together.

[0028] Maximum Euclidean clustering is used to identify and remove point cloud data that does not belong to the shoe body or the whole shoe;

[0029] By converting the base coordinate system to the glue applicator coordinate system, a point cloud of the shoe body or the entire shoe is generated for glue applicator operations, which facilitates the glue applicator robot's operation of the glue applicator process.

[0030] The process of transforming to the base coordinate system using a calibration matrix and then stitching the point clouds together includes:

[0031] The point clouds in other camera coordinate systems are registered with the reference point cloud using a calibration matrix. The position and orientation of other point clouds are iteratively adjusted to align the overlapping parts between point clouds. This process is repeated until all point clouds are integrated into the base coordinate system. Multiple sets of point cloud data are then combined to form a complete shoe body or whole shoe point cloud, and the point cloud stitching is completed.

[0032] The Euclidean clustering method includes calculating the Euclidean distance and comparing it with a preset clustering parameter threshold, identifying noise points and removing useless point clouds.

[0033] Step S3 includes:

[0034] In the base coordinate system, the two sets of shoe body point clouds and the whole shoe point cloud share the point cloud index of the smart pointer, and access the three-dimensional coordinates of the corresponding points of the two;

[0035] Set a distance threshold to search for overlapping areas, with the lower edge as the glue application line;

[0036] Extract the location of the glue line and slice the point cloud of the shoe body. Find the lowest point of each slice as the glue line trajectory for which glue needs to be applied.

[0037] The curve fitting method is a cubic B-spline curve. By adjusting the parameters in the control algorithm, the shape of the curve is changed, so that the fitted curve is smooth and closely resembles the original point cloud data.

[0038] The present invention has the following beneficial effects and advantages:

[0039] A 3D vision-based method for extracting adhesive spraying trajectories on shoe uppers can be used for automated extraction of adhesive lines and adhesive spraying operations in footwear production. This method will significantly improve production efficiency, reduce labor costs, and ensure adhesive quality and product consistency. Furthermore, this method is applicable to various shoe types, including sneakers and athletic shoes, and is not limited to just one type. Attached Figure Description

[0040] Figure 1 This is a flowchart of the method of the present invention;

[0041] Figure 2 A schematic diagram of the complete shoe body point cloud with background noise, stitched together according to this method;

[0042] Figure 3 A schematic diagram of the complete shoe body point cloud after removing useless point clouds;

[0043] Figure 4 A schematic diagram showing the overlapping area found by comparing the point cloud of the complete shoe body with the point cloud of the entire shoe including the sole;

[0044] Figure 5 This is the smooth adhesive line coating trajectory extracted in this invention. Detailed Implementation

[0045] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0046] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The invention will now be described in further detail with reference to the accompanying drawings and embodiments.

[0047] This invention presents a 3D vision-based method for extracting adhesive application trajectories on shoe uppers. A calibration matrix is ​​calculated using the sphere center calibration method. Point clouds captured by four cameras are aggregated together using this calibration matrix to form a complete shoe upper point cloud. Maximum Euclidean clustering is used to extract the shoe upper point cloud from the cluttered point cloud, eliminating the need to adjust parameters based on height values ​​as in direct-pass filtering, thus improving segmentation efficiency. The adhesive line positions are extracted, and the shoe upper point cloud is divided into multiple segments along the x and y directions of the sole length. After slicing, the lowest point of each segment is found and placed into a new point cloud. Combining the point clouds extracted along the x and y directions yields the adhesive line trajectory. B-spline interpolation is used to fit the extracted adhesive line trajectory to the point cloud. The shape of the curve is altered by adjusting the shape of the control points, resulting in a smooth curve that closely approximates the original point cloud data.

[0048] The method for extracting the adhesive spray trajectory of a shoe body based on 3D vision according to the present invention specifically includes the following four steps:

[0049] S1. Steps for calibrating multiple cameras using the sphere center calibration method:

[0050] S101. Data Acquisition: Place the calibration sphere on the support platform, and set up an industrial camera in each of the four directions above the calibration sphere to capture point cloud data of the calibration sphere at the same location from four different perspectives. Change the position of the calibration sphere to trigger the four cameras to capture data again, for a total of point cloud data of the calibration sphere from five different positions under four cameras.

[0051] S102. Filter the point cloud of the calibration sphere and fit a 3D point cloud sphere.

[0052] First, the spherical point cloud needs to be separated from the cluttered point cloud; this includes:

[0053] Use a pass-through filter to set a threshold to remove noticeable cluttered point clouds around the sphere.

[0054] Then, the RANSAC plane fitting principle is used to remove the planar point cloud below the sphere: define the planar model parameters and interior point indices, set the input, and obtain the remaining point cloud after removing the plane.

[0055] Finally, a 3D point cloud sphere needs to be extracted from the remaining point cloud: the model parameters are calculated by randomly selecting data points, and it is determined whether other points conform to the model. This process is iterated repeatedly to find the best-fit sphere.

[0056] S103. Select one of the four cameras as the base coordinate system, calibrate the other three cameras on the base coordinate system, and aggregate the point cloud of the calibration sphere.

[0057] The corresponding spherical point cloud is fitted iteratively using the RANSAC algorithm, its sphere center is calculated, and the calibration matrix (rotation matrix R, translation matrix T) between any camera outside the base coordinate system and the base coordinate system is estimated using SVD.

[0058] The spherical images captured by the four cameras are fitted to the base coordinate system using the transformation matrix described above to obtain a complete sphere, thus completing the calibration.

[0059] S2, Point Cloud Stitching Steps:

[0060] The scanning objects to be processed in this step are the bottomless shoe body and the complete shoe with the sole; the bottomless shoe body and the complete shoe with the sole are respectively placed on the shoe last, and the following processing is performed to obtain the stitched point cloud of the two.

[0061] S201. Using four laser cameras to scan the entire shoe from different angles, point cloud data of the shoe body (whole shoe) under different poses can be obtained. The point cloud of the shoe body (whole shoe) in the base coordinate system is selected as the reference. The point clouds in the coordinate systems of the other cameras are sequentially registered with the reference point cloud using rotation matrix R and translation matrix T. The positions and poses of the other point clouds are iteratively adjusted to align overlapping parts as much as possible. This process is repeated until all point clouds are integrated into the base coordinate system. Multiple sets of point cloud data are then merged into a complete point cloud of the shoe body (whole shoe), completing the point cloud stitching. (See attached image) Figure 1 As shown, a complete dot cloud of the shoe body is displayed, but other messy dot clouds are attached around the shoe body dot cloud.

[0062] S202. Use maximum Euclidean clustering to identify and remove point cloud data that does not belong to the shoe body (whole shoe);

[0063] For Euclidean clustering, Euclidean distance is calculated and compared with a preset clustering parameter threshold as the judgment criterion. The maximum Euclidean clustering method is used to ensure that the point cloud of the entire shoe body (shoe) is completely preserved, while removing useless point clouds such as those from the background and bottom surface. (See attached image.) Figure 2 As shown, the Euclidean clustering algorithm was used to remove useless point clouds, while retaining the complete point cloud of the shoe body.

[0064] S203. Then, the base coordinate system is converted to the glue applicator coordinate system, and finally the point cloud of the shoe body (whole shoe) is generated for glue application, which facilitates the glue application robot to run the glue application process.

[0065] S3. Steps for extracting the adhesive application trajectory:

[0066] S301. Load the processed shoe body point cloud and the whole shoe point cloud including the sole into the same base coordinate system. The two sets of point clouds share the point cloud index in the smart pointer, and access the three-dimensional coordinates (x, y, z) of the corresponding points in the matching point cloud (shoe body point cloud and whole shoe point cloud) through the correspondences class.

[0067] S302. Set the maximum distance between points. Points with a distance less than the maximum distance threshold are considered overlapping points. Find the overlapping area and use the lower edge of this overlapping area as the location of the adhesive line to be applied. (See attached image) Figure 3 As shown, the overlapping area is displayed.

[0068] S303. Extract the glue line location. Divide the shoe body point cloud into multiple segments along the length (x) and width (y) of the sole. After slicing, find the lowest point of each segment and place each lowest point into a new point cloud. The resulting point cloud combination is the glue line trajectory that needs to be applied.

[0069] S304. Sort the extracted adhesive line trajectories into point clouds according to the included angle arctan(y / x) value to obtain a set of ordered adhesive line trajectory point cloud data.

[0070] S4, Smoothing Trajectory Steps:

[0071] Curve fitting is used to fit the sorted point cloud. The shape of the curve is changed by adjusting the shape of the control points, so that the fitted curve is both smooth and closely resembles the original point cloud data. To ensure that the generated trajectory has continuous curvature, the degree of the fitted B-spline curve is at least 3. Therefore, this invention uses a cubic B-spline curve. Fitting parameters, such as curve order and smoothness, are adjusted as needed to achieve the best results. (See attached diagram) Figure 4 As shown, the process of extracting and processing the adhesive line yields a smooth trajectory for applying the adhesive.

[0072] Finally, it should be noted that the above description is a preferred embodiment of the present invention. It should be pointed out that for those skilled in the art, several improvements and modifications can be made without departing from the principles of the present invention, and these improvements and modifications should be considered within the scope of protection of the present invention.

Claims

1. A method for extracting the adhesive spray trajectory of a shoe body based on 3D vision, characterized in that, The following method is used to extract the adhesive spray trajectory on the shoe body. The method includes the following steps: S1. Use the sphere center calibration method to calibrate multiple cameras and obtain the calibration matrix for transformation to the base coordinate system; S2. For the bottomless shoe body and the whole shoe with the sole, obtain the spliced ​​point cloud of the two in the base coordinate system, and delete the interfering and useless points; S3. Under the base coordinate system, access the three-dimensional coordinates of the corresponding points of the matching shoe body point cloud and the whole shoe point cloud, search the overlapping area to extract the glue line, and slice the shoe body point cloud to find the lowest point as the glue line trajectory. S4. The trajectory is smoothed using curve fitting method and used as the glue spraying trajectory for the robot glue spraying process.

2. The method for extracting the adhesive spray trajectory of a shoe body based on 3D vision according to claim 1, characterized in that, The ball center calibration method includes: The calibration sphere is placed under the view of multiple cameras, triggering multiple cameras to simultaneously capture multiple sets of point clouds of the calibration sphere at different positions; The point cloud of the calibration sphere is filtered and then fitted to a 3D point cloud sphere. One of the multiple cameras is selected as the base coordinate system, and the other cameras are calibrated on the base coordinate system. The calibration sphere point cloud is then fitted to obtain the complete calibration sphere point cloud.

3. The method for extracting the adhesive spray trajectory of a shoe body based on 3D vision according to claim 2, characterized in that, The filtration process includes: Extract the spherical point cloud from the cluttered point cloud; Use a pass-through filter to set a threshold to remove cluttered point clouds around the sphere; The RANSAC plane fitting principle is used to remove the planar point cloud below the sphere.

4. The method for extracting the adhesive spray trajectory of a shoe body based on 3D vision according to claim 1, characterized in that, The step of calibrating other cameras on the base coordinate system includes: Calculate the center of the 3D point cloud sphere, and use SVD to calculate the calibration matrix between any camera outside the base coordinate system and the base coordinate system; The images of the sphere captured by the four cameras are fitted to the base coordinate system using the calibration matrix described above to obtain a complete sphere, thus completing the calibration.

5. The method for extracting the adhesive spray trajectory of a shoe body based on 3D vision according to claim 1, characterized in that, Step S2 includes: Multiple cameras are used to scan the shoe body and the whole shoe from different angles to obtain point cloud data of the two in different poses. The calibration matrix is ​​used to transform the data to the base coordinate system and the point clouds of the two are stitched together. Maximum Euclidean clustering is used to identify and remove point cloud data that does not belong to the shoe body or the whole shoe; By converting the base coordinate system to the glue applicator coordinate system, a point cloud of the shoe body or the entire shoe is generated for glue applicator operations, which facilitates the glue applicator robot's operation of the glue applicator process.

6. The method for extracting the adhesive spray trajectory of a shoe body based on 3D vision according to claim 5, characterized in that, The process of transforming to the base coordinate system using a calibration matrix and then stitching the point clouds together includes: The point clouds in other camera coordinate systems are registered with the reference point cloud using a calibration matrix. The position and orientation of other point clouds are iteratively adjusted to align the overlapping parts between point clouds. This process is repeated until all point clouds are integrated into the base coordinate system. Multiple sets of point cloud data are then combined to form a complete shoe body or whole shoe point cloud, and the point cloud stitching is completed.

7. The method for extracting the adhesive spray trajectory of a shoe body based on 3D vision according to claim 5, characterized in that, The Euclidean clustering method includes calculating the Euclidean distance and comparing it with a preset clustering parameter threshold, identifying noise points and removing useless point clouds.

8. The method for extracting the adhesive spray trajectory of a shoe body based on 3D vision according to claim 5, characterized in that, Step S3 includes: In the base coordinate system, the two sets of shoe body point clouds and the whole shoe point cloud share the point cloud index of the smart pointer, and access the three-dimensional coordinates of the corresponding points of the two; Set a distance threshold to search for overlapping areas, with the lower edge as the glue application line; Extract the location of the glue line and slice the point cloud of the shoe body. Find the lowest point of each slice as the glue line trajectory for which glue needs to be applied.

9. The method for extracting the adhesive spray trajectory of a shoe body based on 3D vision according to claim 1, characterized in that, The curve fitting method is a cubic B-spline curve. By adjusting the parameters in the control algorithm, the shape of the curve is changed, so that the fitted curve is smooth and closely resembles the original point cloud data.

Images