Sole glue spraying track extraction method based on 3D vision
By automatically segmenting and repairing the point cloud of the shoe sole using 3D vision technology, the problems of uneven glue application and automation in traditional shoe manufacturing have been solved. This has enabled efficient and accurate extraction of glue application trajectories, improving the coverage and quality of automated shoe manufacturing processes.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CHENYANG ROBOT IND DEVELOPMENT GROUP CO LTD
- Filing Date
- 2024-12-20
- Publication Date
- 2026-06-23
AI Technical Summary
In traditional shoe manufacturing, the sole coating process is prone to problems such as missed coating, glue overflow, coating deviation and unevenness, which leads to a decline in product quality. Moreover, existing technologies are difficult to automate and achieve full-coverage coating.
Using a 3D vision-based approach, techniques such as pass-through filtering, Euclidean clustering, outlier filtering, and moving least squares are employed to automatically segment and repair the point cloud of the shoe sole, extract reasonable adhesive application trajectories, and achieve full coverage.
It improves the automation and coverage of sole adhesive spraying, ensures the accuracy and uniformity of adhesive application, and optimizes the efficiency and quality of the shoe manufacturing process.
Smart Images
Figure CN122265129A_ABST
Abstract
Description
Technical Field
[0001] This invention pertains to image processing and point cloud visual computing, specifically to the automatic extraction of adhesive spraying trajectories on shoe soles in automated shoe production lines. The method for extracting adhesive spraying trajectories on shoe soles is based on 3D vision. Background Technology
[0002] In traditional adhesive coating processes, frequent instances of missed areas and excess adhesive during sole coating are due to human error. This not only weakens the strong adhesion of the sole but also directly affects product quality. Excessive adhesive application can also lead to excess adhesive buildup on the shoe edges, impacting appearance. Furthermore, the uncontrollable nature of manual operation, such as coating deviations caused by hand tremors and the difficulty in ensuring uniformity, becomes a bottleneck restricting production efficiency and quality improvement. This results in health risks, efficiency limitations, and human error inherent in manual operations. Existing sole trajectory extraction solutions suffer from the following problems:
[0003] 1) Based on the point cloud image of the shoe sole captured by the 3D vision camera, the point cloud on the side of the shoe sole may be partially missing due to occlusion, lighting and other reasons, which will affect the subsequent trajectory extraction.
[0004] 2) The segmentation of the shoe sole point cloud is mostly obtained by manual cutting, which makes it impossible to automate the entire extraction process.
[0005] 3) It is difficult to calculate the actual spraying points on the sole, making it impossible to achieve the ideal effect of full glue coverage.
[0006] Therefore, with the continued development of the footwear industry, there is a need for an effective method to solve the problem of adhesive spraying on shoe soles. Summary of the Invention
[0007] In view of the above-mentioned defects and shortcomings of the existing technology, the purpose of this invention is to provide a method for extracting shoe sole adhesive application trajectory based on 3D vision. The method fills in the point cloud based on the distribution characteristics of the shoe sole point cloud and methods such as moving least squares, addressing situations where the shoe sole point cloud is missing. Euclidean clustering is used to segment the shoe sole point cloud, avoiding manual point cloud trimming and thus automating the process. Normal vector edge extraction, bias rules, and a full coverage algorithm are used to obtain a reasonable shoe sole adhesive application trajectory to achieve full adhesive coverage.
[0008] The technical solution adopted by the present invention to achieve the above objectives is as follows:
[0009] A 3D vision-based method for extracting the adhesive spray trajectory on the shoe sole involves the following steps:
[0010] S1. Use a camera to collect the original point cloud containing the shoe sole, and process it sequentially through the pass-through filter method, Euclidean clustering method, and outlier filtering algorithm to initially segment the shoe sole point cloud from the whole image;
[0011] S2. Determine whether point cloud repair is needed. If so, based on the distribution characteristics of the shoe sole point cloud, combine clustering methods to initially fill the point cloud. Based on the initial filling, use moving least squares method for smoothing and resampling to achieve point cloud repair; otherwise, skip step S2 and proceed to step S3.
[0012] S3. Calculate the normal vector and curvature of the data points using a spherical range as the scale, determine the edge boundary of the upper sole point cloud, and remove non-boundary points through clustering; based on the coarsely extracted upper sole point cloud boundary, offset inward to obtain the offset curve adhesive trajectory.
[0013] S4. Based on the upper bias curve trajectory points of the shoe sole point cloud, the trajectory is projected onto a two-dimensional plane through coordinate transformation to offset the curve inward, and multiple sets of full-coverage points are obtained by slicing, and then sorted to obtain the straight adhesive application trajectory.
[0014] The step of segmenting the shoe sole point cloud includes:
[0015] Construct a pass-through filter for cutting the original point cloud, and filter out useless data points by defining the distance threshold of the camera in the depth direction by setting the parameters of the pass-through filter;
[0016] Euclidean clustering is used to group points that are close together into the same cluster, thus extracting the shoe sole point cloud from the whole image;
[0017] Accurate shoe sole point cloud is obtained by using an outlier filtering algorithm that can remove noise.
[0018] The point cloud repair steps include:
[0019] Based on the distribution characteristics of the shoe sole side, an outlier filtering method is used to process the data to overcome the fact that the point cloud information on the shoe sole side is significantly smaller than that in other areas; combined with Euclidean clustering, the shoe sole point cloud is split into an upper annular point cloud and a lower curved surface point cloud.
[0020] The spatial points of the point cloud along the side of the shoe sole are supplemented in a linear form; for the tiny holes in the point cloud of the shoe sole, the moving least squares (MLS) method is used for upsampling to fill the holes and smooth the surface.
[0021] For the upsampled shoe sole point cloud, voxel filtering is used to filter out duplicate points.
[0022] The linear spatial point supplementation includes:
[0023] The kd-tree method is used to construct a search tree. Each point in the upper point cloud is set as a search point to find its corresponding point in the lower point cloud. This is done by supplementing the spatial points in a linear fashion to achieve the initial repair of the shoe sole point cloud.
[0024] The moving least squares (MLS) method is used to fill the gap by setting key parameters of the MLS method, namely the search radius and the interpolation point radius.
[0025] The curve trajectory extraction includes:
[0026] The kd-tree method is used to construct a search tree in the upper point cloud, and points whose spatial normal vectors and curvatures meet the threshold requirements are calculated and filtered using a spherical range as the scale, and the upper edge boundary is roughly extracted.
[0027] Euclidean clustering was used to further refine the upper edge boundary of the sole, the boundary points were arranged in an ordered manner according to the large-scale voxel filtering method, and spline interpolation was used to smooth the curve.
[0028] The obtained curve is biased inward in three-dimensional space, and the biased curve is smoothed by spline to obtain the curve of the glue spraying trajectory of the point cloud on the upper layer of the shoe sole.
[0029] The inward bias is:
[0030] Based on each adjacent pair of points p on the trajectory n (x n ,y n ,z n ),p n+1 (x n+1 ,y n+！ ,z n+1 Constructing spatial vectors Based on p n and Construct a spatial plane and set the plane interval distance. Slice the shoe sole point cloud into multiple sets and then slice the trajectory points p. n The bias is set based on the slice to which it belongs, serving as the starting point.
[0031] The step of projecting the trajectory onto a two-dimensional plane through coordinate transformation and then offsetting the curve inward includes:
[0032] The PCA method is used to process the curve and transform the offset curve to a new coordinate system;
[0033] Preserve the z-value information in the depth direction, project the point cloud onto the xy two-dimensional plane, and calculate the direction vector of each point to be offset inward; offset each point at an equal distance along the perpendicular vector pointing to the centroid to obtain a new boundary line;
[0034] Based on the new boundary line, multiple sets of equidistant slices are processed along the non-principal axis direction, and the intersection of the slices and the new boundary line is taken as the endpoint of the straight glue spraying trajectory.
[0035] The two endpoints of the obtained range line are sorted according to rules to finally obtain the straight adhesive spray trajectory that fully covers the sole.
[0036] The present invention has the following beneficial effects and advantages:
[0037] 1. The method of the present invention integrates a highly automated segmentation module and a filling module for filling missing point clouds, and uses slicing, offset and other techniques to accurately and reasonably extract spraying trajectory points. At the same time, it allows flexible parameter settings and combinations of curved and straight adhesive spraying trajectories, which greatly improves the coverage and accuracy of adhesive spraying on shoe soles, and optimizes the rationality and practicality.
[0038] 2. This invention is applicable to most common shoe sole configurations on the market and can help achieve efficient and precise automated shoe manufacturing processes. Attached Figure Description
[0039] Figure 1 This is a schematic diagram of the method flow of the present invention.
[0040] Figure 2 A schematic diagram of the original point cloud acquired by the vision module and the point cloud of the shoe sole output by the cutting module.
[0041] Figure 3 This diagram illustrates the entire process of reconstructing the shoe sole point cloud using the point cloud repair module; it includes two processes: straight line repair and MLS upsampling.
[0042] Figure 4 This is a schematic diagram of the feature points on the upper edge of the sole and their offset trajectories obtained through normal vectors and curvature;
[0043] Figure 5 This is a schematic diagram of the result obtained by using the innermost offset curve as the boundary line and the straight line trajectory extraction module. Detailed Implementation
[0044] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.
[0045] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The invention will now be described in further detail with reference to the accompanying drawings and embodiments.
[0046] like Figure 1The diagram shown is a schematic representation of the method flow of this invention. The method uses a camera to acquire images, obtaining an original point cloud containing the shoe sole, and then performs the following specific processing steps:
[0047] S1. Point cloud segmentation steps:
[0048] S101. Based on the physical characteristics of the visual usage scenario, a pass-through filter for cutting the initial point cloud is first constructed. The distance threshold of the camera in the depth direction is defined by setting the parameters of the pass-through filter. The threshold parameter is 800mm-900mm. This step can filter out most useless data points, including the surrounding background and equipment, and it works stably based on the unchanged scene.
[0049] S102. Use Euclidean clustering to segment the shoe sole point cloud. The principle of this method is based on the Euclidean distance between points in the point cloud, with a threshold parameter of 2mm. Points that are close to each other are grouped into the same cluster, thereby achieving the segmentation and clustering extraction of the shoe sole point cloud from the whole image.
[0050] S103. Finally, an outlier filtering algorithm that can remove noise is used, with a threshold parameter of 20 neighboring points and a standard deviation multiple of 0.5, to obtain a more concise and accurate shoe sole point cloud.
[0051] like Figure 2 The image shows a schematic diagram of the original point cloud acquired by vision and the shoe sole point cloud output by the segmentation module. In the image, the x-direction is horizontal to the right, the y-direction is vertical to the up, and the z-direction is the image depth direction, i.e. the shoe sole thickness direction. As can be seen from the figure, this step can segment the shoe sole point cloud.
[0052] S2. Determine whether the shoe sole point cloud result segmented in step S1 needs to be repaired. Whether point cloud repair is needed is determined by whether the number of point clouds with the same density is less than the fixed percentage parameter of the complete template point cloud. The parameter is 90%. If it is needed, perform point cloud repair according to the following steps. Otherwise, skip step S2 and proceed to step S3.
[0053] S201. During the acquisition of shoe sole point cloud data, the 3D camera should be relatively perpendicular to the shoe sole and maintained at a certain height. This will result in a significantly smaller amount of point cloud information along the side of the shoe sole compared to other areas. An outlier filtering rule is formulated based on the distribution characteristics, with a threshold parameter of 50 neighboring points and a standard deviation factor of 1, to process the shoe sole point cloud. Based on this, Euclidean clustering is used to split the shoe sole point cloud into two parts: an upper annular point cloud and a lower curved surface point cloud.
[0054] S202. Use the kd-tree method to construct a search tree, set each point in the upper point cloud as a search point to find its corresponding point in the lower point cloud, and implement the point supplementation in a linear form in a one-to-many manner to achieve the initial repair of the shoe sole point cloud.
[0055] For the repaired shoe sole point cloud containing small holes, MLS (Moving Least Squares) is used for filling. The principle is to fit a local surface (such as a plane or curved surface) by calculating a local weighted average in the point cloud. Each point is adjusted according to other points in its neighborhood and a new point is generated, thereby achieving upsampling of the point cloud data. At the same time, it can fill holes and smooth surfaces in the point cloud. When using MLS, it is necessary to pay attention to the setting of two key parameters: search radius and interpolation point radius. The threshold parameters are 5mm for search radius and 3mm for interpolation point radius.
[0056] S203. For the upsampled shoe sole point cloud, voxel filtering is finally used to filter out duplicate points. The threshold parameter is a voxel cube with a side length of 0.5 mm to avoid affecting the efficiency of subsequent calculations.
[0057] like Figure 3 This diagram illustrates the entire process of reconstructing the shoe sole point cloud through point cloud repair, including two processes: straight line repair and MLS upsampling.
[0058] S3. When the quality of the shoe sole point cloud obtained in step S1 is good enough, or after the repair in step S2 is completed, proceed with the curve trajectory extraction step:
[0059] S301. Use the kd-tree method to construct a search tree in the upper point cloud, and calculate the spatial normal vector and curvature of each point on a spherical scale; extract points with large changes in normal vector and curvature based on the PCL (Point Cloud Library) boundary recognition library, and use a boundary judgment angle threshold of 0.83π as the threshold parameter to obtain a rough extraction of the upper edge boundary.
[0060] S302. The upper edge boundary of the sole is obtained by using Euclidean clustering, large-scale voxel filtering is performed, and the boundary points are sorted in clockwise order. Finally, spline interpolation is used to smooth the curve.
[0061] S303. Apply an inward offset to the obtained curve in three-dimensional space. Specifically, based on each pair of adjacent points p on the trajectory... n (x n ,y n ,z n ),p n+1 (x n+1 ,y n+！ ,z n+1 Constructing spatial vectors Based on p n and Construct a spatial plane and set the plane interval distance. Slice the shoe sole point cloud into multiple sets and then slice the trajectory points p. n Starting from the slice, an offset is applied based on the slice thickness of 2mm. The offset rule is not fixed; it can be a single offset at a fixed distance or multiple offsets at accumulated distances. The number of offset curves and the offset distance can be set according to the actual situation. After the offset curve is smoothed by spline, the curve of the glue spraying trajectory of the upper point cloud of the shoe sole can be obtained.
[0062] like Figure 4 As shown, this is a schematic diagram of the feature points on the upper edge of the sole and their offset trajectories obtained through normal vectors and curvature. The diagram shows two offset curves.
[0063] S4. For one or more curve trajectories obtained, select the innermost offset line as the boundary line of the straight line trajectory, and extract the planned straight adhesive spray trajectory within the range.
[0064] S401. First, the curve is processed using PCA (Principal Component Analysis) to obtain the corresponding eigenvectors v1, v2, and v3. The three eigenvectors can be used to construct a new coordinate system direction. The coordinate system expressed by the reconstructed matrix is processed according to the actual situation, and the curve point cloud located in the original coordinate system is transformed to the new coordinate system.
[0065] S402. Retaining the z-value information of each point in the depth direction, project the point cloud onto the xy two-dimensional plane and calculate the inward offset direction vector for each point. Considering the relationship between the constructed vector and its perpendicular vector of two ordered adjacent points in a two-dimensional problem (i.e., the two coordinate values are exchanged, and one is inverted), the computational load is reduced. By offsetting each point equidistantly along its perpendicular vector pointing to the centroid, a new boundary line can be obtained.
[0066] S403. Based on the new boundary line, perform multiple sets of equidistant slices along the non-main axis direction, i.e., the leg length direction, with adjustable slice intervals of 5mm. The intersection of the slices and the new boundary line is taken as the endpoint of the straight adhesive spraying trajectory.
[0067] S404. The two endpoints of the obtained range line are sorted according to a rule to obtain a straight-line adhesive spray trajectory that fully covers the sole. The rule sorting can be a straight-line trajectory in the shape of an arc, a zigzag, or other shapes.
[0068] like Figure 5 The diagram shows the result obtained by using the innermost offset curve as the boundary line and the straight trajectory extraction module. The direction of motion in the diagram can be indicated by the arrows.
[0069] This invention uses coarse scene-based direct-pass filtering and precise Euclidean clustering to extract shoe sole point clouds from raw point clouds, replacing manual cropping and automating the entire trajectory extraction process. Based on the camera's shooting position and point cloud distribution characteristics, MLS, line-filling rules, and clustering are used to handle missing parts of the point cloud. Unlike mesh surface reconstruction, this approach significantly improves computational efficiency. By constructing the plane to which the ordered points on the upper edge belong and performing planar slicing, a specified number of curve trajectories can be obtained through a single distance offset or cumulative distance offsets, improving the overall rationality of trajectory extraction. By projecting the innermost offset curve and offsetting the plane inward, the effective boundary of the straight-line trajectory is reasonably set, and a specified group of slice points is obtained and sorted based on the planar slices, greatly increasing the trajectory coverage.
[0070] Finally, it should be noted that the above description is a preferred embodiment of the present invention. It should be pointed out that for those skilled in the art, several improvements and modifications can be made without departing from the principles of the present invention, and these improvements and modifications should be considered within the scope of protection of the present invention.
Claims
1. A method for extracting the trajectory of adhesive spraying on shoe soles based on 3D vision, characterized in that, The following method is used to extract the adhesive spray trajectory on the shoe sole. The method includes the following steps: S1. Use a camera to collect the original point cloud containing the shoe sole, and process it sequentially through the pass-through filter method, Euclidean clustering method, and outlier filtering algorithm to initially segment the shoe sole point cloud from the whole image; S2. Determine whether point cloud repair is needed. If so, based on the distribution characteristics of the shoe sole point cloud, combine clustering methods to initially fill the point cloud. Based on the initial filling, use moving least squares method for smoothing and resampling to achieve point cloud repair; otherwise, skip step S2 and proceed to step S3. S3. Calculate the normal vector and curvature of the data points using a spherical range as the scale, determine the edge boundary of the upper sole point cloud, and remove non-boundary points through clustering; based on the coarsely extracted upper sole point cloud boundary, offset inward to obtain the offset curve adhesive trajectory. S4. Based on the upper bias curve trajectory points of the shoe sole point cloud, the trajectory is projected onto a two-dimensional plane through coordinate transformation to offset the curve inward, and multiple sets of full-coverage points are obtained by slicing, and then sorted to obtain the straight adhesive application trajectory.
2. The method for extracting shoe sole adhesive spray trajectory based on 3D vision according to claim 1, characterized in that, The step of segmenting the shoe sole point cloud includes: Construct a pass-through filter for cutting the original point cloud, and filter out useless data points by defining the distance threshold of the camera in the depth direction by setting the parameters of the pass-through filter; Euclidean clustering is used to group points that are close together into the same cluster, thus extracting the shoe sole point cloud from the whole image; Accurate shoe sole point cloud is obtained by using an outlier filtering algorithm that can remove noise.
3. The method for extracting the adhesive spray trajectory of shoe soles based on 3D vision according to claim 2, characterized in that, The point cloud repair steps include: Outlier filtering is used to process the shoe sole side cloud based on its distribution characteristics to avoid low information content in the shoe sole side cloud; Euclidean clustering is combined to split the shoe sole point cloud into an upper annular point cloud and a lower curved surface point cloud. The spatial points of the point cloud along the side of the shoe sole are supplemented in a linear form; for the tiny holes in the point cloud of the shoe sole, the moving least squares (MLS) method is used for upsampling to fill the holes and smooth the surface. For the upsampled shoe sole point cloud, voxel filtering is used to filter out duplicate points.
4. The method for extracting shoe sole adhesive spraying trajectory based on 3D vision according to claim 3, characterized in that, The linear spatial point supplementation includes: The kd-tree method is used to construct a search tree. Each point in the upper point cloud is set as a search point to find its corresponding point in the lower point cloud. This is done by supplementing the spatial points in a linear fashion to achieve the initial repair of the shoe sole point cloud.
5. The method for extracting shoe sole adhesive spraying trajectory based on 3D vision according to claim 3, characterized in that, The moving least squares (MLS) method is used to fill the gap by setting key parameters of the MLS method, namely the search radius and the interpolation point radius.
6. The method for extracting shoe sole adhesive spray trajectory based on 3D vision according to claim 3, characterized in that, The curve trajectory extraction includes: The kd-tree method is used to construct a search tree in the upper point cloud, and points whose spatial normal vectors and curvatures meet the threshold requirements are calculated and filtered using a spherical range as the scale, and the upper edge boundary is roughly extracted. Euclidean clustering was used to further refine the upper edge boundary of the sole, the boundary points were arranged in an ordered manner according to the large-scale voxel filtering method, and spline interpolation was used to smooth the curve. The obtained curve is biased inward in three-dimensional space, and the biased curve is smoothed by spline to obtain the curve of the glue spraying trajectory of the point cloud on the upper layer of the shoe sole.
7. The method for extracting shoe sole adhesive spray trajectory based on 3D vision according to claim 6, characterized in that, The inward bias is: Based on each adjacent pair of points p on the trajectory n (x n ,y n ,z n ),p n+1 (x n+1 ,y n+！ ,z n+1 Constructing spatial vectors Based on p n and Construct a spatial plane and set the plane interval distance. Slice the shoe sole point cloud into multiple sets and then slice the trajectory points p. n The bias is set based on the slice to which it belongs, serving as the starting point.
8. The method for extracting shoe sole adhesive spray trajectory based on 3D vision according to claim 6, characterized in that, The step of projecting the trajectory onto a two-dimensional plane through coordinate transformation and then offsetting the curve inward includes: The PCA method is used to process the curve and transform the offset curve to a new coordinate system; Preserve the z-value information in the depth direction, project the point cloud onto the xy two-dimensional plane, and calculate the direction vector of each point to be offset inward; offset each point at an equal distance along the perpendicular vector pointing to the centroid to obtain a new boundary line; Based on the new boundary line, multiple sets of equidistant slices are processed along the non-principal axis direction, and the intersection of the slices and the new boundary line is taken as the endpoint of the straight glue spraying trajectory. The two endpoints of the obtained range line are sorted according to rules to finally obtain the straight adhesive spray trajectory that fully covers the sole.