High-heeled shoe sole glue spraying track extraction method

Abstract

The disclosure provides a high-heeled shoe sole glue spraying track extraction method, which comprises the following steps: collecting point cloud data of different angles of the same high-heeled shoe, and reconstructing a three-dimensional point cloud model of the high-heeled shoe through a point cloud registration algorithm; performing model matching on a three-dimensional design model of the high-heeled shoe production and manufacturing and the three-dimensional point cloud model after registration to obtain a design glue spraying track; mapping the design glue spraying track in the three-dimensional space to a two-dimensional plane through a manifold; performing Kalman filtering processing on the point clouded high-heeled shoe sole 2D model or the collected shoe sole three-dimensional point cloud and the glue spraying track point cloud on the two-dimensional plane to obtain a two-dimensional plane glue spraying track; performing spatial transformation on the two-dimensional plane glue spraying track to convert it into a predicted three-dimensional point cloud track in the high-heeled shoe glue coating surface, and outputting the predicted glue spraying track as a predicted glue spraying track; and performing optimal second Kalman filtering optimization on the design glue spraying track and the predicted glue spraying track to obtain an optimal glue spraying track.

Description

Technical Field

[0001] This disclosure relates to a method and apparatus for extracting the adhesive spray trajectory of high heel soles, and more particularly to a method for extracting the adhesive spray trajectory of complex curved surfaces of high heel soles based on 3D vision. Background Technology

[0002] Replacing manual glue application with robots can save production costs for enterprises, improve the quality of glue spraying on shoe soles, and avoid the harm to human health caused by toxic gases emitted from glue volatilization. Industrial robots not only have great flexibility but also high production efficiency. The use of robots for glue application on shoe soles has been adopted by many shoe manufacturing companies. However, due to the difficulty in extracting the glue spraying trajectory of high heels, robots have been difficult to apply to the automated glue spraying of high heels.

[0003] Currently, significant progress has been made in the research of extracting adhesive spray trajectories from shoe soles. Existing methods for extracting adhesive spray trajectories can be divided into two categories: those based on 2D image processing and those based on 3D vision. Compared to 2D image processing, 3D vision can obtain richer object pose information and has been widely used in extracting adhesive spray trajectories from shoe soles on complex curved surfaces.

[0004] Existing shoe sole adhesive trajectory extraction technologies are generally based on the shoe edge contour captured by a camera. When extracting adhesive trajectories on the complex curved surfaces of high heels, it is easy to misjudge the adhesive trajectory. Summary of the Invention

[0005] To address one of the aforementioned technical problems, this disclosure provides a method for extracting the adhesive spray trajectory on the sole of a high-heeled shoe.

[0006] According to one aspect of this disclosure, a method for extracting the adhesive spray trajectory of a high-heeled shoe sole is provided, comprising:

[0007] Point cloud data of the same high heel from different angles were collected, and a three-dimensional point cloud model of the high heel was reconstructed using a point cloud registration algorithm.

[0008] Model matching is performed between the 3D design model and the registered 3D point cloud model for high heel shoe manufacturing to obtain the designed glue spraying trajectory;

[0009] The designed adhesive spraying trajectory in 3D space is mapped onto a 2D plane using a manifold; the point cloud-based 2D model of the high-heeled shoe sole or the collected 3D point cloud of the sole and the adhesive spraying trajectory point cloud on the 2D plane are processed by Kalman filtering to obtain the 2D plane adhesive spraying trajectory; the 2D plane adhesive spraying trajectory is spatially transformed to convert it into a predicted 3D point cloud trajectory in the adhesive coating surface of the high-heeled shoe, which is then used as the output of the predicted adhesive spraying trajectory.

[0010] The designed glue spraying trajectory and the predicted glue spraying trajectory are optimized by optimal quadratic Kalman filtering to obtain the optimal glue spraying trajectory, which is then used as the glue spraying trajectory followed by the robot's glue spraying process.

[0011] According to at least one embodiment of the high-heeled shoe sole adhesive trajectory extraction method of the present disclosure, the model matching of the three-dimensional design model of high-heeled shoe manufacturing and the registered three-dimensional point cloud model includes: matching the reconstructed three-dimensional point cloud model with the point cloudified high-heeled shoe design model.

[0012] According to at least one embodiment of the high-heeled shoe sole adhesive spraying trajectory extraction method of the present disclosure, the reconstructed three-dimensional point cloud model is matched with the point cloud-based high-heeled shoe design model to obtain the model design adhesive spraying curve, and according to the model design adhesive spraying curve, the corresponding adhesive spraying trajectory in the high-heeled shoe point cloud model is generated by KNN nearest neighbor search, and the corresponding adhesive spraying trajectory in the point cloud model is used as the design adhesive spraying trajectory.

[0013] According to at least one embodiment of the high-heeled shoe sole adhesive trajectory extraction method of the present disclosure, the method of mapping the designed adhesive trajectory in three-dimensional space to a two-dimensional plane through manifold mapping includes: transforming the point cloud of the designed adhesive trajectory into a point cloud on a two-dimensional plane through spatial transformation, performing Kalman filtering optimization on the two-dimensional curved surface of the spatial transformation and the shoe sole point cloud, and mapping the extracted optimized two-dimensional plane adhesive trajectory to the space where the high-heeled shoe curved surface is located through manifold mapping, and outputting it as the predicted adhesive point cloud contour. Attached Figure Description

[0014] The accompanying drawings illustrate exemplary embodiments of the present disclosure and, together with the description thereof, serve to explain the principles of the present disclosure. These drawings are included to provide a further understanding of the present disclosure and are incorporated in and constitute a part of this specification.

[0015] Figure 1 This is a flowchart of a method for extracting the adhesive spray trajectory of a high-heeled shoe sole according to one embodiment of the present disclosure.

[0016] Figure 2 This is the result of mapping the high-heeled shoe sole spray adhesive trajectory to a two-dimensional plane under controlled distortion conditions in the high-heeled shoe sole spray adhesive trajectory extraction method according to one embodiment of the present disclosure.

[0017] Figure 3 This is the result of the optimal adhesive spray trajectory of a high-heeled shoe sole spray trajectory extraction method according to one embodiment of the present disclosure.

[0018] Figure 4 The result is the optimal adhesive spray trajectory of a high-heeled shoe sole spray trajectory extraction method according to an embodiment of the present disclosure, after processing by Bézier curve interpolation. Detailed Implementation

[0019] The present disclosure will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the disclosure. Furthermore, it should be noted that, for ease of description, only the parts relevant to the present disclosure are shown in the accompanying drawings.

[0020] It should be noted that, where there is no conflict, the embodiments and features described in this disclosure can be combined with each other. The technical solutions of this disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0021] Unless otherwise stated, the exemplary implementations / embodiments shown are to be understood as providing exemplary features of various details that provide ways in which the technical concepts of this disclosure can be implemented in practice. Therefore, unless otherwise stated, the features of various implementations / embodiments may be additionally combined, separated, interchanged and / or rearranged without departing from the technical concepts of this disclosure.

[0022] The use of crosshairs and / or shading in the accompanying drawings is generally used to clarify the boundaries between adjacent components. Thus, unless otherwise stated, the presence or absence of crosshairs or shading does not convey or indicate any preference or requirement for the specific material, material properties, dimensions, proportions, commonalities between the illustrated components, or any other characteristics, properties, etc., of the components. Furthermore, in the accompanying drawings, the dimensions and relative dimensions of components may be exaggerated for clarity and / or descriptive purposes. When exemplary embodiments can be implemented differently, a specific process sequence may be performed in a different order than that described. For example, two consecutively described processes may be performed substantially simultaneously or in the reverse order of their description. Furthermore, the same reference numerals denote the same components.

[0023] When a component is referred to as being "on" or "above" another component, "connected to," or "joined to" another component, the component may be directly on, directly connected to, or directly joined to the other component, or there may be intermediate components. However, when a component is referred to as being "directly on" another component, "directly connected to," or "directly joined to" another component, there are no intermediate components. Therefore, the term "connection" can refer to a physical connection, an electrical connection, etc., and may or may not have intermediate components.

[0024] For descriptive purposes, this disclosure may use spatial relative terms such as “below,” “under,” “below,” “down,” “above,” “above,” “higher,” and “side (e.g., in a “sidewall”)” to describe the relationship between one component and another component as shown in the accompanying drawings. In addition to the orientations depicted in the drawings, the spatial relative terms are also intended to encompass different orientations of the device during use, operation, and / or manufacture. For example, if the device in the drawings is flipped, a component described as “below” or “under” another component or feature would subsequently be positioned “above” said other component or feature. Thus, the exemplary term “below” can encompass both “above” and “below” orientations. Furthermore, the device may be otherwise positioned (e.g., rotated 90 degrees or in other orientations), thus interpreting the spatial relative descriptive terms used herein accordingly.

[0025] The terminology used herein is for the purpose of describing particular embodiments and is not intended to be limiting. As used herein, unless the context clearly indicates otherwise, the singular forms “a” and “the” are intended to include the plural forms as well. Furthermore, when the terms “comprising” and / or “including” and variations thereof are used in this specification, it indicates the presence of the stated features, integrals, steps, operations, parts, components, and / or groups thereof, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, parts, components, and / or groups thereof. It should also be noted that, as used herein, the terms “substantially,” “about,” and other similar terms are used as approximate terms rather than as terms of degree, thus explaining the inherent biases in measurements, calculated values, and / or provided values ​​that would be recognized by one of ordinary skill in the art.

[0026] Figure 1 This is a flowchart of a method for extracting the adhesive spray trajectory of a high-heeled shoe sole according to one embodiment of the present disclosure.

[0027] In this disclosure, the method for extracting the adhesive spray trajectory of high-heeled shoe soles can be implemented using a high-heeled shoe sole adhesive spray trajectory extraction device. This device comprises an optical lidar camera, a multi-degree-of-freedom robotic arm, an external axis, an industrial control computer, and tooling fixtures. The optical lidar camera includes a color camera module, a line laser module, a processor module, a power supply module, and mechanical structural components.

[0028] The multi-degree-of-freedom manipulator specifically includes: a 4-axis or 5-axis CNC system-controlled manipulator, a 6-joint robot, etc. Hand-eye calibration is performed using calibration plates and blocks to calibrate the coordinate relationship between the multi-degree-of-freedom manipulator and the optical lidar.

[0029] like Figure 1As shown, the high-heeled shoe sole adhesive spraying trajectory extraction method disclosed herein includes: S1, collecting point cloud data of the same high heel from different angles, and reconstructing a three-dimensional point cloud model of the high heel through a point cloud registration algorithm; S2, performing model matching between the three-dimensional design model of high heel manufacturing and the registered three-dimensional point cloud model to obtain the designed adhesive spraying trajectory; S3, mapping the designed adhesive spraying trajectory in three-dimensional space onto a two-dimensional plane through manifold mapping; extracting the point cloud-based 2D model of the high heel sole or performing Kalman filtering on the collected three-dimensional point cloud of the sole and the adhesive spraying trajectory point cloud on the two-dimensional plane to obtain the two-dimensional plane adhesive spraying trajectory; performing spatial transformation on the two-dimensional plane adhesive spraying trajectory to convert it into a predicted three-dimensional point cloud trajectory in the adhesive coating surface of the high heel, as the predicted adhesive spraying trajectory output; S4, performing optimal quadratic Kalman filtering optimization on the designed adhesive spraying trajectory and the predicted adhesive spraying trajectory to obtain the optimal adhesive spraying trajectory, and using the optimal adhesive spraying trajectory as the adhesive spraying trajectory followed by the robot adhesive spraying process.

[0030] Therefore, the high-heeled shoe sole adhesive spraying trajectory extraction method disclosed herein can extract the adhesive spraying trajectory of various styles of high heels and execute the spraying trajectory by a robot, thereby reducing labor costs. In particular, the present disclosure uses visual guidance, so that the accuracy of the adhesive spraying trajectory is not affected by factors such as shoe upper deformation and lasting, and has a wide range of applications.

[0031] The above steps will be explained in detail below with reference to specific implementation methods.

[0032] In S1, point cloud data of high heels at different angles overlaid on the shoe last are acquired, and a complete 3D point cloud model of the high heels is reconstructed through ICP point cloud registration, while only the point cloud of the high heels is retained.

[0033] In a specific embodiment, S1 specifically includes:

[0034] The high heels are fitted onto the shoe last, and the pose of the high heels is changed through an external axis. The camera collects point cloud data of the high heels from different angles.

[0035] Each collected point cloud data is segmented, retaining only the point cloud of the high heels, and point cloud registration is performed to reconstruct the complete high heel point cloud. The high heel point cloud is then preprocessed, including downsampling, noise reduction, and denoising.

[0036] Specifically, the shoe upper is fitted onto the shoe last, with uppers of different sizes fitted onto the corresponding shoe lasts. The external axis is rotated via a terminal device to change the high heel's pose, and the camera acquires point cloud data of the high heel in different poses. Each captured point cloud image is segmented, retaining the point cloud portion belonging to the high heel. The segmented point cloud data is then used for ICP point cloud registration to reconstruct the complete high heel point cloud data.

[0037] Point cloud images of high heels in different poses are acquired using an optical lidar system and input into a pre-designed PointNet++ point cloud segmentation model. The high heel portion of the point cloud is segmented, and the segmented point clouds from different areas are then registered using ICP (Internal Point Registration) to reconstruct a complete high heel point cloud model. ICP registration involves searching for feature points in each input point cloud image and matching these feature points across different point cloud images. The registration is based on minimizing the Euclidean distance between identical feature points in different point cloud images, resulting in a rotation matrix R between the different point clouds. * Translation matrix T * The objective function is:

[0038]

[0039] Where: P S and P t This refers to inputting two point clouds that need to be registered. and These are the corresponding feature points, and R and t are the rotation and translation matrices generated during the registration process. By adjusting the registered point cloud feature points... and The optimal rotation matrix R is obtained by minimizing the overall Euclidean distance. * Translation matrix T * As output.

[0040] The reconstructed high-heeled shoe point cloud model A1 is preprocessed using the following steps:

[0041] The point cloud is downsampled to retain the high heel feature information while reducing the data volume. Then, the point cloud is subjected to radius filtering to remove noise points, resulting in point cloud A2. Point cloud A2 is then used to replace the reconstructed high heel point cloud model A1.

[0042] In this disclosure, S2 specifically involves: converting the 3D design model of the high heels into point clouds and matching it with the reconstructed point cloud model of the high heels to obtain the design adhesive spraying trajectory.

[0043] Specifically, the point cloud-based design model is matched with the reconstructed 3D point cloud model, the reconstructed 3D point cloud model is converted to the coordinate system of the design model, and the two are merged to achieve the optimal degree of overlap.

[0044] Extract the preset glue spraying trajectory from the design model, search for its corresponding path in the reconstructed high-heeled shoe point cloud model using KNN nearest neighbor search, detect outliers in the glue spraying trajectory, remove outliers, and output the designed glue spraying trajectory points after sequential processing.

[0045] The KNN nearest neighbor search is based on minimizing the Euclidean distance between the search origin P(xa,ya,za) and the search point P(xb,yb,zb).

[0046]

[0047] In S3, the point cloud of the shoe sole to be bonded is mapped onto the reconstructed point cloud surface of the high heel. The PRNet point cloud registration algorithm is used to register the shoe sole and the high heel to maximize their overlap.

[0048] In a specific embodiment, S3 specifically includes: transforming the acquired design adhesive spraying trajectory into a two-dimensional plane through Riemannian manifold mapping while controlling distortion. The manifold mapping treats the design adhesive spraying trajectory P and the bonding sole plane R as manifold surfaces in Euclidean space, respectively. By constructing a mapping φ, every point on manifold P is mapped to manifold R, i.e., φ: P → R.

[0049] The two-dimensional adhesive spraying curve is registered with the point cloud of the bonded sole. A cost function that minimizes the sum of distances between the adhesive spraying curve and the boundary points of the sole is constructed. A predicted adhesive spraying trajectory is generated by Kalman filtering and mapped back to the adhesive spraying surface as the output of the predicted adhesive spraying trajectory.

[0050] Extract the boundary contour point cloud of the sole after registration with the high heel, process it sequentially, and then generate the glue spraying trajectory through offset processing. Perform KDtree-based nearest neighbor search on the offset glue spraying path in the high heel point cloud to obtain the robot's predicted glue spraying path D in the high heel as the output.

[0051] The offset processing disclosed herein includes the following steps: After obtaining the robot's predicted glue spraying trajectory D, a point in its point cloud contour is P0(x0,y0), and its corresponding offset point to be determined is Q(xq,yq). Given an offset distance d(u), first search for the nearest point P1(x1,y2) in the point cloud D, using P0 as the search point. straight line The normal vector is Therefore, the bias point Q q (x q ,y q )for:

[0052]

[0053]

[0054]

[0055] S4 includes: performing optimal quadratic Kalman filtering on the output designed glue spraying trajectory and the predicted glue spraying trajectory to generate the optimal glue spraying trajectory for the robot; detecting abnormal points in the high-heeled shoe glue spraying trajectory, removing abnormal points, and performing Bézier curve interpolation on the glue spraying trajectory to generate more transition points between points with different distances from the trajectory points, thus smoothing the glue spraying trajectory; generating robot shoe upper glue spraying instructions based on the glue spraying process parameters, and controlling the robot and the glue spraying chamber to spray glue onto the shoe upper according to the robot shoe upper glue spraying instructions.

[0056] Kalman filtering is a state-optimal estimation algorithm that uses the designed glue spraying trajectory and the predicted glue spraying trajectory as the predicted values, respectively. With the measured value x k The optimal glue spraying trajectory path is obtained from the input system using the following formula:

[0057]

[0058] Among them, P k represent Compared with the estimated value x k The covariance, H is x k The state observation matrix is ​​given by K, where K is the Kalman gain, representing the ratio of model prediction error to measurement error in the optimal state estimation process. K ∈ [0,1]. When K = 0, the prediction error is 0, and the system's state value depends entirely on the predicted value. When K = 1, the measurement error is 0, and the system's state value depends entirely on the measured value. The estimation principle of Kalman filtering is to minimize the covariance P of the optimal state estimate. k The goal is to minimize it, making it increasingly closer to the true value. Therefore, its objective function is:

[0059] J = ∑ min P k

[0060] For example, the point cloud of the designed glue spraying trajectory and the point cloud of the predicted glue spraying trajectory are optimized to calibrate the glue spraying path of the high heels and obtain the optimal glue spraying trajectory of the high heels. The obtained optimal glue spraying path is then interpolated using Bézier curves to increase the transition points between each path point, thereby obtaining a smooth glue spraying path. This smooth glue spraying path is then used as the optimal glue spraying estimate for the robot to follow when spraying glue.

[0061] More specifically, given points P0, P1, ..., P n Its Bézier curve is:

[0062]

[0063] In the formula:

[0064]

[0065] In the description of this specification, the references to terms such as "one embodiment / mode," "some embodiments / modes," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment / mode or example is included in at least one embodiment / mode or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment / mode or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments / modes or examples. Furthermore, without contradiction, those skilled in the art can combine and integrate the different embodiments / modes or examples described in this specification, as well as the features of different embodiments / modes or examples.

[0066] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0067] Those skilled in the art should understand that the above embodiments are merely for illustrating the present disclosure and are not intended to limit the scope of the disclosure. Those skilled in the art can make other changes or modifications based on the above disclosure, and these changes or modifications still fall within the scope of the present disclosure.

Claims

1. A method for extracting the adhesive spray trajectory on the sole of a high-heeled shoe, characterized in that, include: Point cloud data of the same high heel from different angles were collected, and a three-dimensional point cloud model of the high heel was reconstructed using a point cloud registration algorithm. Model matching is performed between the 3D design model and the registered 3D point cloud model for high heel shoe manufacturing to obtain the designed glue spraying trajectory; The designed adhesive spraying trajectory in 3D space is mapped onto a 2D plane using a manifold; the point cloud-based 2D model of the high-heeled shoe sole or the collected 3D point cloud of the sole and the adhesive spraying trajectory point cloud on the 2D plane are processed by Kalman filtering to obtain the 2D plane adhesive spraying trajectory; the 2D plane adhesive spraying trajectory is spatially transformed to convert it into a predicted 3D point cloud trajectory in the adhesive coating surface of the high-heeled shoe, which is then used as the output of the predicted adhesive spraying trajectory. The designed glue spraying trajectory and the predicted glue spraying trajectory are optimized by optimal quadratic Kalman filtering to obtain the optimal glue spraying trajectory, which is then used as the glue spraying trajectory followed by the robot's glue spraying process.

2. The method for extracting the adhesive spray trajectory of high-heeled shoe soles according to claim 1, characterized in that, Model matching between the 3D design model for high-heel shoe manufacturing and the registered 3D point cloud model includes: matching the reconstructed 3D point cloud model with the point cloud-based high-heel shoe design model.

3. The method for extracting the adhesive spray trajectory of high-heeled shoe soles according to claim 2, characterized in that, The reconstructed 3D point cloud model is matched with the point cloud-based high-heeled shoe design model to obtain the model's design glue spraying curve. Based on the model's design glue spraying curve, the corresponding glue spraying trajectory in the high-heeled shoe point cloud model is generated through KNN nearest neighbor search, and the corresponding glue spraying trajectory in the point cloud model is used as the design glue spraying trajectory.

4. The method for extracting the adhesive spray trajectory of high-heeled shoe soles according to claim 1, characterized in that, Mapping the designed adhesive spraying trajectory in three-dimensional space to a two-dimensional plane via manifolds includes: transforming the point cloud of the designed adhesive spraying trajectory into a point cloud on a two-dimensional plane through spatial transformation, performing Kalman filtering optimization on the two-dimensional curved surface and the shoe sole point cloud of the spatial transformation, and then mapping the extracted optimized two-dimensional plane adhesive spraying trajectory to the space where the high-heeled shoe surface is located through manifolds, and outputting it as the predicted adhesive spraying point cloud contour.

Images