A method, system, device and program product for detecting the quality of shoe upper lasting

Abstract

The present application belongs to the technical field of shoe quality detection, and aims to provide a shoe upper lasting quality detection method, system, device and program product. In the implementation process of the present application, shoe upper scanning data of a to-be-detected shoe upper is first obtained; then a template shoe upper data set matched with the to-be-detected shoe upper is extracted from a preset shoe upper template database, and to-be-detected feature point position information of to-be-detected feature points on the to-be-detected shoe upper is obtained according to the shoe upper scanning data and the template shoe upper data set; wherein the template shoe upper data set includes template feature values; subsequently, detection feature values corresponding to the to-be-detected feature points are obtained according to the to-be-detected feature point position information; finally, a quality detection result of the to-be-detected shoe upper is obtained according to the detection feature values and the template feature values. The present application can reduce the labor cost of shoe upper lasting quality detection, has high detection efficiency, and can improve detection accuracy.

Description

Technical Field

[0001] This invention belongs to the field of footwear quality inspection technology, specifically relating to a method, system, equipment, and procedure for inspecting the quality of shoe uppers and lasts. Background Technology

[0002] China is a major shoe manufacturing country, accounting for approximately 55% of the world's shoe production. In the shoe manufacturing process, the upper-lasting process is a crucial step in ensuring shoe quality. Specifically, this process involves precisely fitting the upper to the shoe last to ensure a tight fit and form the basic shape of the shoe. Currently, the vast majority of shoe manufacturing processes are still completed manually. After the upper-lasting process, quality inspection is typically performed manually by personnel responsible for the quality control process.

[0003] However, in using the prior art, the inventors discovered at least the following problems:

[0004] On the one hand, manual inspection requires labor costs, resulting in high labor costs. On the other hand, manual inspection usually lacks precise quantitative standards and mainly relies on visual confirmation by users. However, the upper of a shoe is a three-dimensional structure, and the deviation value of human visual perception is relatively large, which makes manual inspection prone to false detections and missed detections. This largely leads to poor consistency in finished shoes and affects the overall quality of footwear. Summary of the Invention

[0005] The present invention aims to solve the above-mentioned technical problems to at least a certain extent. The present invention provides a method, system, equipment and program product for quality inspection of shoe upper lasts.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] In a first aspect, the present invention provides a method for detecting the quality of shoe upper lasting, comprising:

[0008] Obtain the upper scan data of the shoe upper to be tested;

[0009] Extract template shoe upper datasets that match the shoe upper to be tested from a preset shoe upper template database, and obtain the location information of the test feature points on the shoe upper to be tested based on the shoe upper scanning data and the template shoe upper dataset; wherein, the template shoe upper dataset includes template feature values;

[0010] Based on the location information of the feature point to be tested, the detection feature value corresponding to the feature point to be tested is obtained;

[0011] The quality inspection result of the shoe upper to be tested is obtained based on the detection feature value and the template feature value.

[0012] This invention reduces the labor cost of quality inspection of shoe uppers and lasts, achieving high inspection efficiency and improving inspection accuracy. Specifically, in the implementation process, the invention first acquires the shoe upper scanning data of the shoe upper to be tested; then, it extracts a template shoe upper dataset matching the shoe upper to be tested from a preset shoe upper template database, and obtains the location information of the feature points to be tested on the shoe upper based on the shoe upper scanning data and the template shoe upper dataset; wherein, the template shoe upper dataset includes template feature values; subsequently, based on the location information of the feature points to be tested, the detection feature values ​​corresponding to the feature points to be tested are obtained; finally, based on the detection feature values ​​and the template feature values, the quality inspection result of the shoe upper to be tested is obtained. In this process, the invention realizes automatic quality inspection of shoe uppers and lasts, with low labor costs, high inspection efficiency, and improved inspection accuracy. Practice has proven that the inspection accuracy using this invention can reach 0.5mm, and it also facilitates the timely rework of substandard shoe upper semi-finished products, thereby improving the overall shoe production yield.

[0013] In one possible design, the template shoe upper dataset also includes basic parameters of the template shoe upper; correspondingly, a template shoe upper dataset matching the shoe upper to be tested is extracted from a preset shoe upper template database, including:

[0014] Obtain the basic parameters of the shoe upper to be tested;

[0015] Based on the basic parameters of the upper of the shoe to be tested, a template upper dataset with the same basic parameters as the upper of the shoe to be tested is retrieved from the template upper dataset, and this template upper dataset is used as the template upper dataset that matches the upper of the shoe to be tested.

[0016] In one possible design, the upper scanning data is obtained by scanning the outer surface of the upper by circling the upper around it. The upper scanning data includes three-dimensional point cloud data and two-dimensional image data of the upper.

[0017] Correspondingly, the template shoe upper dataset includes template shoe upper scanning data and template feature point location information. The template shoe upper scanning data includes 3D point cloud template data and 2D image template data. Based on the shoe upper scanning data and the template shoe upper dataset, the location information of the feature points to be tested on the shoe upper to be tested is obtained, including:

[0018] Point cloud matching is performed on the 3D point cloud data and the 3D point cloud template data to obtain the transformed 3D point cloud data with the smallest error between them and the 3D point cloud template data; wherein, the transformed 3D point cloud data is obtained by performing point cloud transformation on the 3D point cloud data;

[0019] Based on the 3D point cloud template data and the 2D image template data, obtain the template feature descriptor of the template feature point corresponding to the template feature point position information;

[0020] Using the template feature point location information of any template feature point as the center of a sphere and the region inside the sphere with radius r as the search area, the test feature descriptors of all test points in the transformed 3D point cloud data located within the search area are obtained; where r > 0;

[0021] The feature descriptors of each test point are compared with the template feature descriptors in turn, and the test point with the smallest error with the template feature descriptors is selected as the test feature point of the shoe upper.

[0022] The three-dimensional coordinate data of the feature point to be tested is extracted from the transformed three-dimensional point cloud data, and the three-dimensional coordinate data is used as the position information of the feature point to be tested on the shoe upper.

[0023] In one possible design, point cloud matching is performed between the 3D point cloud data and the 3D point cloud template data, including:

[0024] Obtain the optimal transformation matrix between the 3D point cloud data and the 3D point cloud template data;

[0025] The three-dimensional point cloud data is transformed according to the optimal transformation matrix to obtain the transformed three-dimensional point cloud data; wherein, the error between the transformed three-dimensional point cloud data and the three-dimensional point cloud template data is minimized.

[0026] The optimal transformation matrix includes an optimal rotation matrix and an optimal translation matrix; correspondingly, the error between the transformed 3D point cloud data and the 3D point cloud template data is:

[0027]

[0028] In the formula, P′ refers to the transformed 3D point cloud data; Q refers to the 3D point cloud template data; R refers to the optimal rotation matrix; t refers to the optimal translation matrix; p i q refers to the three-dimensional coordinate data of the i-th point in the shoe upper scan data; i The term refers to the 3D coordinate template data of the i-th point in the template shoe upper scanning data; n is the total number of the 3D point cloud data and the 3D point cloud template data.

[0029] In one possible design, based on the 3D point cloud template data and the 2D image template data, a template feature descriptor for the template feature points corresponding to the template feature point location information is obtained, including:

[0030] Based on the 3D point cloud template data and the 2D image template data, obtain the k nearest neighbor sample points to the template feature points, as well as the 3D coordinate data and color data of the k sample points; where k is a natural number greater than 0.

[0031] Pair the k sample points with the template feature points to obtain k(k+1) / 2 pairs of points.

[0032] Based on the 3D point cloud template data, a local uvw coordinate system is constructed for each pair of points; the coordinate values ​​in the local uvw coordinate system for any pair of points are as follows: u = n s , w = y × v; where p t and p s These are the three-dimensional coordinate data of the two points in the current point pair; n s To match the three-dimensional coordinate data p s The normal vector of the corresponding point; × indicates the cross product sign;

[0033] Using the coordinate values ​​in each local UVW coordinate system, the feature description vector of each point in the corresponding point pair is obtained; where, compared with the three-dimensional coordinate data p s The description parameters in the feature description vector of the corresponding point are:

[0034] α=v·n t ;

[0035]

[0036] θ = tan -1 (w·n s ,u·n s );

[0037] d = ||p t -p s || 2 ;

[0038] γ=‖c t -c s || 2 ;

[0039] In the formula, α, δ, θ, d, and γ all refer to descriptive parameters. α, δ, and θ represent the angular transformation relationship between the coordinate system formed by the line connecting two points and the base coordinate system. d is the Euclidean distance between two points in the local uvw coordinate system. γ is the Euclidean distance between the color data of two points in the local uvw coordinate system. · represents the inner product symbol. t c is the actual physical distance between the two points corresponding to the center of the local uvw coordinate system; sThe color difference value between two points corresponding to the points in the local uvw coordinate system;

[0040] Based on the feature description vectors of all points in the point set, the template feature points are encoded to obtain the template feature descriptors of the template feature points.

[0041] In one possible design, multiple feature points to be measured are set, and the detection feature value corresponding to each feature point includes the distance between two specified feature points to be measured; correspondingly, the distance between two specified feature points to be measured is obtained by:

[0042] Obtain the position information of two specified feature points to be tested corresponding to two specified feature points to be tested, and construct a tangent plane based on the position information of the two specified feature points to be tested; wherein, the tangent plane is a plane formed by the straight line connecting the two specified feature points to be tested and the normal vectors corresponding to the two specified feature points to be tested;

[0043] The intersection of the cutting plane with the three-dimensional point cloud data of the shoe upper to be tested yields the intersection line segment;

[0044] The intersecting line segments are discretized to obtain multiple discretized line segments;

[0045] Obtain the lengths of multiple discrete line segments, and sum the lengths of the multiple discrete line segments to obtain the distance between two specified feature points to be measured.

[0046] In one possible design, if the quality inspection result of the shoe upper to be tested is unqualified, the unqualified location information is output to a preset shoe upper marking device, so that the shoe upper marking device can mark the position on the shoe upper to be tested corresponding to the unqualified location information.

[0047] Secondly, the present invention provides a shoe upper last quality inspection system for implementing the shoe upper last quality inspection method as described in any of the above claims; the shoe upper last quality inspection system includes:

[0048] The scanning data acquisition module is used to acquire the scanning data of the upper of the shoe to be tested;

[0049] The location information calculation module is communicatively connected to the scanning data acquisition module. It is used to extract a template shoe upper dataset that matches the shoe upper to be tested from a preset shoe upper template database, and to obtain the location information of the test feature points on the shoe upper to be tested based on the shoe upper scanning data and the template shoe upper dataset. The template shoe upper dataset includes template feature values.

[0050] The feature value calculation module is communicatively connected to the scanning data acquisition module and is used to obtain the detection feature value corresponding to the feature point to be tested based on the location information of the feature point to be tested.

[0051] The detection result acquisition module is communicatively connected to the feature value calculation module and is used to obtain the quality detection result of the shoe upper to be tested based on the detection feature value and the template feature value.

[0052] Thirdly, the present invention provides an electronic device, comprising:

[0053] Memory, used to store computer program instructions; and,

[0054] A processor is configured to execute the computer program instructions to perform the operation of the shoe upper last quality inspection method as described in any of the above.

[0055] Fourthly, the present invention provides a computer program product, including a computer program or instructions, wherein the computer program or instructions, when executed by a computer, implement the shoe upper last quality inspection method as described in any of the above. Attached Figure Description

[0056] Figure 1 This is a flowchart of the shoe upper last quality inspection method in Example 1;

[0057] Figure 2 This is a schematic diagram of the shoe upper data acquisition device in Example 1, which performs shoe upper scanning data acquisition on the shoe upper to be tested.

[0058] Figure 3 This is a visual diagram of the shoe upper scanning data obtained in Example 1;

[0059] Figure 4 This is a visual schematic diagram of the shoe upper scanning data in Example 1, which identifies the feature values ​​of the example template;

[0060] Figure 5 This is a schematic diagram of point cloud data displayed after global spatial alignment of the three-dimensional point cloud data and the three-dimensional point cloud template data in Embodiment 1;

[0061] Figure 6 This is a visual schematic diagram of the shoe upper scanning data with the cutting plane marked in Example 1;

[0062] Figure 7 This is a schematic diagram of the structure of the shoe upper last quality detection device for implementing the shoe upper last quality detection method in Embodiment 1;

[0063] Figure 8 This is a block diagram of the shoe upper last quality inspection system in Example 2. Detailed Implementation

[0064] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the present invention will be briefly introduced below in conjunction with the accompanying drawings and descriptions of the embodiments or the prior art. Obviously, the following description of the structure of the accompanying drawings is only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. It should be noted that the description of these embodiments is for the purpose of helping to understand the present invention, but does not constitute a limitation of the present invention.

[0065] Example 1:

[0066] This embodiment discloses a method for quality inspection of shoe upper lasts, which can be executed by, but is not limited to, a computer device or virtual machine with certain computing resources, such as a personal computer, smartphone, personal digital assistant or wearable device, or by a virtual machine.

[0067] like Figure 1 As shown, a method for quality inspection of shoe upper lasting may include, but is not limited to, the following steps:

[0068] S1. Obtain the upper scanning data of the shoe upper to be tested; In this embodiment, the upper scanning data is obtained by scanning the outer surface of the outer ring of the shoe upper to be tested by circling the shoe upper to be tested along a path. The circling path can be a rectangular path of the outer ring of the shoe upper to be tested, which is not limited here. Using a rectangular path facilitates subsequent point coordinate calculations. The upper scanning data includes the three-dimensional point cloud data and two-dimensional image data of the shoe upper to be tested. It should be understood that in this embodiment, the three-dimensional point cloud data of the shoe upper is acquired by a line laser three-dimensional camera in a preset upper data acquisition device, and the two-dimensional image data of the shoe upper is acquired by a line scan two-dimensional camera in a preset upper data acquisition device. The line scan two-dimensional camera carries a line scan light source, and the upper data acquisition device can rotate around the outer ring of the shoe upper under the drive of a preset rotating and moving device. As an example... Figure 2 A schematic diagram of the shoe upper data acquisition device is given, which collects shoe upper scanning data from the shoe upper under test. In the figure, 101 refers to the line laser 3D camera, 102 refers to the line scan 2D camera, and 16 refers to the shoe upper under test. Figure 3 A visualization diagram of the scanned shoe upper data is provided.

[0069] In addition, before collecting the shoe upper scanning data, the line laser 3D camera used to collect the 3D point cloud data of the shoe upper and the line scan 2D camera used to collect the 2D image data of the shoe upper are jointly calibrated so that the shoe upper scanning data can be jointly represented by the 3D point cloud data and the 2D image data. The joint calibration is achieved by existing technology and will not be described in detail here.

[0070] Specifically, in this embodiment, the shoe upper scanning data can be represented as PC, where PC = {p1, c1, p2, c2, ..., p i ,c i ,…,p n ,c n}; where n is the total number of the three-dimensional point cloud data and the two-dimensional image data; p i p represents the three-dimensional coordinate data of the i-th point in the shoe upper scan data. i =(x i ,y i ,z i ), x i y i and z i These are the three-dimensional position coordinates of the i-th point in the shoe upper scan data, that is, the x-axis or horizontal axis coordinate, y-axis or vertical axis coordinate, and z-axis or vertical axis coordinate with the specified point as the reference; c i c is the color data of the i-th point in the shoe upper scan data. i =(r i ,g i ,b i ), r i g i and b i These are the pixel values ​​of the RGB three channels of the i-th point in the shoe upper scan data.

[0071] S2. Extract template shoe upper datasets that match the shoe upper to be tested from a preset shoe upper template database, and obtain the location information of the test feature points on the shoe upper to be tested based on the shoe upper scanning data and the template shoe upper dataset; wherein, the template shoe upper dataset includes template feature values;

[0072] Specifically, in this embodiment, the shoe upper template database is constructed before the quality inspection of the shoe upper to be tested. The shoe upper template database pre-stores template shoe upper datasets that match various shoe uppers to be tested. The template shoe upper dataset includes basic parameters of the template shoe upper, template shoe upper scanning data, template feature point location information, and template feature values, etc. The basic parameters of the template shoe upper include information such as shoe type, size, and left / right foot identification; the template feature point location information includes location information such as the center axis position of the toe and the center axis position of the heel. For example, such as... Figure 4As shown, the template feature values ​​include the distance between the toe center axis and the tangent point of the sole to the toe pattern, the distance between the logo on the side of the upper and the tangent point of the sole, the distance 161 between the heel pattern of the upper and the tangent point of the sole, the distance 162 between the left heel pattern of the upper and the tangent point of the sole (also known as the height of the left heel pattern of the upper) and / or the distance 163 between the right heel pattern of the upper and the tangent point of the sole (also known as the height of the right heel pattern of the upper), etc., which are not limited here.

[0073] The template shoe upper dataset also includes basic parameters of the template shoe upper; correspondingly, in step S2, a template shoe upper dataset matching the shoe upper to be tested is extracted from a preset shoe upper template database, including:

[0074] Obtain the basic parameters of the shoe upper to be tested;

[0075] Based on the basic parameters of the upper of the shoe to be tested, a template upper dataset with the same basic parameters as the upper of the shoe to be tested is retrieved from the template upper dataset, and this template upper dataset is used as the template upper dataset that matches the upper of the shoe to be tested.

[0076] In this embodiment, the template shoe upper dataset includes template shoe upper scanning data and template feature point location information. The template shoe upper scanning data includes 3D point cloud template data and 2D image template data. In step S2, based on the shoe upper scanning data and the template shoe upper dataset, the location information of the feature points to be tested on the shoe upper to be tested is obtained, including:

[0077] S201. Perform point cloud matching on the three-dimensional point cloud data and the three-dimensional point cloud template data to obtain the transformed three-dimensional point cloud data with the smallest error between it and the three-dimensional point cloud template data; wherein, the transformed three-dimensional point cloud data is obtained by performing point cloud transformation on the three-dimensional point cloud data.

[0078] Specifically, in step S201, point cloud matching is performed on the 3D point cloud data and the 3D point cloud template data, including:

[0079] S201a. Obtain the optimal transformation matrix between the 3D point cloud data and the 3D point cloud template data;

[0080] S201b. The three-dimensional point cloud data is transformed according to the optimal transformation matrix to obtain the transformed three-dimensional point cloud data; wherein, the error between the transformed three-dimensional point cloud data and the three-dimensional point cloud template data is minimized;

[0081] Specifically, in this embodiment, the ICP (Iterative Closest Point) algorithm can be used, but is not limited to, to achieve point cloud registration between the 3D point cloud data and the 3D point cloud template data, so as to align the global space of the 3D point cloud data and the 3D point cloud template data into a unified coordinate system, thereby obtaining the optimal transformation matrix between the 3D point cloud data and the 3D point cloud template data. The ICP algorithm continuously improves the alignment effect between the 3D point cloud data and the 3D point cloud template data through iteration, and has advantages such as simplicity, ease of implementation, and low dependence on initial pose. As an example, this embodiment... Figure 5 A schematic diagram of point cloud data after global spatial alignment of the 3D point cloud data and the 3D point cloud template data is given, wherein the light-colored part of the point cloud data 171 refers to the 3D point cloud template data, and the dark-colored part of the point cloud data 172 refers to the 3D point cloud data.

[0082] The optimal transformation matrix includes the optimal rotation matrix R and the optimal translation matrix t; correspondingly, the error between the transformed 3D point cloud data and the 3D point cloud template data is:

[0083]

[0084] In the formula, P′ refers to the transformed three-dimensional point cloud data, which can be represented as P={p1,p2,…,p i ,…,p n}, then P′=RP-t; Q refers to the three-dimensional point cloud template data; R refers to the optimal rotation matrix; t refers to the optimal translation matrix; p i q refers to the three-dimensional coordinate data of the i-th point in the shoe upper scan data; i The term refers to the 3D coordinate template data of the i-th point in the template shoe upper scanning data; n is the total number of the 3D point cloud data and the 3D point cloud template data.

[0085] It should be noted that in this embodiment, the 3D point cloud data undergoes point cloud transformation according to the optimal transformation matrix. That is, the 3D point cloud data is rotated according to the optimal rotation matrix R and then translated according to the optimal translation matrix t to obtain the transformed 3D point cloud data. The error between the transformed 3D point cloud data and the 3D point cloud template data is minimized, that is, the sum of the distances between the 3D coordinate data of all points in the transformed 3D point cloud data P and the 3D coordinate template data of all corresponding points in the 3D point cloud template data Q is minimized.

[0086] S202. Based on the three-dimensional point cloud template data and the two-dimensional image template data, obtain template feature descriptors for template feature points corresponding to the template feature point location information; wherein, multiple template feature points are provided; in this embodiment, by using a color-combined feature descriptor description method, the location of feature points can be quickly and easily realized.

[0087] Specifically, in step S202, based on the three-dimensional point cloud template data and the two-dimensional image template data, a template feature descriptor corresponding to the template feature point position information is obtained, including:

[0088] S202a. Based on the 3D point cloud template data and the 2D image template data, obtain the k nearest neighbor sample points to the template feature point, as well as the 3D coordinate data and color data of the k sample points; where k is a natural number greater than 0; it should be understood that the k nearest neighbor sample points to the template feature point are sample points whose distance from the template feature point is less than a preset value; the 2D image template data includes color data of multiple sample points, and the color data corresponding to the k sample points in the 2D image template data can be obtained based on the 3D coordinate data of the k sample points;

[0089] S202b. Pair the k sample points and the template feature point into k(k+1) / 2 pairs. Specifically, assuming k is 3, select one sample point from the k sample points (3 choices), then select another sample point from the remaining sample points (2 choices). Therefore, the number of point pair combinations is 3*2 = 6. However, there are duplicate selections of sample points A+B and B+A in the above 6 point pair combinations, so the actual number of point pair combinations is 3*2 / 2 = 3. Similarly, if there are k+1 points including k sample points and one template feature point, the number of point pair combinations is (k+1)*k / 2. In this embodiment, pairing two points is used to construct a sufficient number of point pairs to describe the template feature point, so that the feature descriptor of the template feature point is unique, which helps to improve the accuracy of subsequent detection of the feature point.

[0090] S202c. Based on the 3D point cloud template data, construct a local uvw coordinate system for each pair of points; the coordinate values ​​in the local uvw coordinate system for any pair of points are as follows: u = n s , w = u × v; where p t and p s These are the three-dimensional coordinate data of the two points in the current point pair; n s To match the three-dimensional coordinate data p s The normal vector of the corresponding point; × indicates the cross product sign;

[0091] S202d. Using the coordinate values ​​in each local uvw coordinate system, obtain the feature description vector of each point in the corresponding point pair; where, compared with the three-dimensional coordinate data p s The description parameters in the feature description vector of the corresponding point are:

[0092] α=v·n t ;

[0093]

[0094] θ = tan -1 (w·n s ,u·n s );

[0095] d = ||p t -p s || 2 ;

[0096] γ=‖c t -c s || 2 ;

[0097] In the formula, α, δ, θ, d, and γ all refer to descriptive parameters, which are user-defined settings. α, δ, and θ represent the angular transformation relationship between the coordinate system formed by the line connecting two points and the base coordinate system; d is the Euclidean distance between two points in the local UVW coordinate system; and γ is the Euclidean distance between the color data of two points in the local UVW coordinate system. · represents the inner product symbol; c t c is the actual physical distance between the two points corresponding to the center of the local uvw coordinate system; s The color difference value between two points corresponding to the points in the local uvw coordinate system;

[0098] S202e. Encode the template feature points based on the feature description vectors of all points in the point set to obtain the template feature descriptors of the template feature points.

[0099] Specifically, in this embodiment, by placing the feature description vectors of the point pairs of the template feature points in a histogram, and dividing each feature description vector into 3 intervals, the feature vector of the template feature point has 243 dimensions. Since the first three descriptions in the feature description vector are angles, the feature vector of the template feature point can be normalized. By voting on the feature description vector of each point pair according to its interval, the feature description vector of the template feature point, i.e., the template feature descriptor, can be obtained. The template feature descriptor is represented as follows:

[0100] F t ={N1,N2,N3,…,N 243};

[0101] In the formula, N1, N2, N3, ..., N 243 All of these are feature vectors of the template feature points.

[0102] S203. Using the template feature point position information of any template feature point as the center of a sphere and the region inside the sphere with radius r as the search region, obtain the test feature descriptors of all test points in the transformed 3D point cloud data located within the search region; where r > 0; specifically, in this embodiment, the radius r is determined based on the position of all template feature points in the template shoe upper. To minimize the computational load, in this embodiment, the search region is a spherical region with the smallest volume, and all template feature points are located within the search region; in this embodiment, the process of obtaining the test feature descriptors of the test points is the same as the process of obtaining template feature descriptors, and will not be described again here.

[0103] S204. The feature descriptors of each test point are compared sequentially with the template feature descriptors, and the test point with the smallest error between the test point and the template feature descriptor is selected as the test feature point of the shoe upper to be tested. It should be understood that the smaller the error between the feature descriptor of any test point and the template feature descriptor, the closer the position of the test point is to the position of the template feature point corresponding to the template feature descriptor. In addition, in this embodiment, multiple template feature points are set for each test feature point. Specifically, in this embodiment, the feature descriptors of each test point are compared one by one with multiple template feature descriptors, and the feature point with the smallest error between each of the multiple template feature descriptors is selected as the test feature point of the shoe upper to be tested.

[0104] Specifically, in this embodiment, the error between the feature descriptor of the i-th test point and the template feature descriptor is:

[0105] e(F t ,F i )=‖F t -F i || 2 ;

[0106] In the formula, F t F represents the template feature descriptor; i Let i represent the feature descriptor for the i-th test point.

[0107] S205. Extract the three-dimensional coordinate data of the feature point to be tested from the transformed three-dimensional point cloud data, and use the three-dimensional coordinate data as the position information of the feature point to be tested on the shoe upper.

[0108] It should be noted that in this embodiment, the three-dimensional point cloud data and the three-dimensional point cloud template data are first matched to obtain the transformed three-dimensional point cloud data, so that the transformed three-dimensional point cloud data and the three-dimensional point cloud template data are located in an area where point cloud comparison can be easily performed, thereby facilitating the obtaining of the position information of the feature points to be tested on the transformed three-dimensional point cloud data of the shoe upper to be tested based on the position information of the template feature points in the template shoe upper dataset.

[0109] Furthermore, it should be noted that a feature descriptor is a vector representation that describes a given key point (such as a corner point, edge point, or other feature point). Its purpose is to extract the feature information of the key point for tasks such as image matching and target recognition. In this embodiment, to facilitate the rapid retrieval of the test feature point in the transformed 3D point cloud data corresponding to the template feature point position on the template shoe upper, a template feature descriptor for the template feature point is established, and a search area is defined to obtain the test feature descriptors of all test points in the transformed 3D point cloud data located within the search area. Finally, by comparing the test feature descriptors of each test point with the template feature descriptor in sequence, the test feature point position information of the test feature point on the shoe upper is obtained. In this process, since this embodiment obtains the test feature point position information based on feature descriptors, the search speed for the test feature point is faster and the search accuracy is higher, and the search for the test feature point can be automated.

[0110] S3. Based on the location information of the feature point to be tested, obtain the detection feature value corresponding to the feature point to be tested.

[0111] In this embodiment, multiple feature points to be measured are provided, and the detection feature value corresponding to each feature point includes the distance between two specified feature points to be measured; correspondingly, in step S3, obtaining the distance between two specified feature points to be measured includes:

[0112] S301. Obtain the position information of two specified feature points to be tested, and construct a tangent plane based on the position information of the two specified feature points to be tested; wherein, the tangent plane is a plane formed by the straight line connecting the two specified feature points to be tested and the normal vectors corresponding to the two specified feature points to be tested; for example... Figure 6 As shown, 181 is an all-plane.

[0113] S302. Intersect the cutting plane with the three-dimensional point cloud data of the shoe upper to be tested to obtain the intersection line segment;

[0114] S303. Discretize the intersecting line segments to obtain multiple discrete line segments;

[0115] S304. Obtain the lengths of multiple discrete line segments, and sum the lengths of the multiple discrete line segments to obtain the distance between two specified feature points to be measured.

[0116] It should be noted that, based on the above steps S301 to S304, the distance between the two feature points to be tested along the surface of the shoe upper can be obtained, rather than the straight-line distance in space, which can facilitate further improvement in the accuracy of shoe upper quality inspection.

[0117] It should be understood that the detection feature values ​​corresponding to the feature points to be measured may also include data such as parallelism and / or angle, which are not restricted here.

[0118] S4. Based on the detection feature value and the template feature value, the quality inspection result of the shoe upper to be tested is obtained.

[0119] Specifically, in this embodiment, the quality inspection results include qualified and unqualified quality; correspondingly, in step S4, the quality inspection results of the shoe upper to be tested are obtained based on the detection feature value and the template feature value, including:

[0120] The detected feature value is compared with the template feature value to obtain a comparison result, which is the difference between the detected feature value and the template feature value. If the absolute value of the comparison result is not greater than a preset value, the quality of the shoe upper to be tested is determined to be qualified, and a quality test result of qualified quality is obtained. If the absolute value of the comparison result is greater than the preset value, the quality of the shoe upper to be tested is determined to be unqualified, and a quality test result of unqualified quality is obtained.

[0121] In this embodiment, if the quality inspection result of the shoe upper to be tested is unqualified, the unqualified location information is output to the preset shoe upper marking device so that the shoe upper marking device can mark the position on the shoe upper to be tested corresponding to the unqualified location information.

[0122] It should be understood that, in the implementation of this embodiment, the shoe upper to be tested, including the shoe upper to be tested, is placed sequentially on multiple shoe upper fixing devices preset on the shoe upper conveying device. The shoe upper data acquisition device for scanning and collecting shoe upper data, and the shoe upper marking device for marking the shoe upper when the shoe upper quality is unqualified, are both located within the travel range of the shoe upper conveying device. Furthermore, during the process of conveying the shoe upper, the shoe upper first passes through the shoe upper data acquisition device and then passes through the shoe upper marking device.

[0123] like Figure 7 The diagram shown is a structural schematic of the shoe upper last quality detection device 1 that implements the shoe upper last quality detection method in this embodiment; Figure 7The diagram illustrates the shoe upper data acquisition device 10, the shoe upper fixing device 11, the shoe upper conveying device 12, the shoe upper marking device 13, and the shoe upper to be tested 14.

[0124] In this embodiment, by outputting the information of the defective location to the preset shoe upper marking device, the defective location of the shoe upper to be tested can be marked, which makes it easier for users to quickly repair the shoe upper, thereby further improving shoe manufacturing efficiency.

[0125] Furthermore, in this embodiment, after obtaining the quality inspection result of the shoe upper to be tested, the shoe upper can be transported to the unloading position by the shoe upper conveying device for manual unloading, and subsequent processing can be carried out according to the quality inspection result, which will not be elaborated here.

[0126] This embodiment reduces the labor cost of quality inspection of shoe uppers and lasts, achieving high inspection efficiency and improving inspection accuracy. Specifically, in the implementation process, the shoe upper scanning data of the shoe upper to be tested is first acquired; then, a template shoe upper dataset matching the shoe upper to be tested is extracted from a preset shoe upper template database, and the location information of the feature points to be tested on the shoe upper is obtained based on the shoe upper scanning data and the template shoe upper dataset; wherein, the template shoe upper dataset includes template feature values; subsequently, the detection feature value corresponding to the feature point to be tested is obtained based on the location information of the feature point to be tested; finally, the quality inspection result of the shoe upper to be tested is obtained based on the detection feature value and the template feature value. In this process, this embodiment realizes automatic inspection of the quality of shoe uppers and lasts, with low labor costs, high inspection efficiency, and improved inspection accuracy. Practice has proven that the inspection accuracy using this embodiment can reach 0.5mm, and it also facilitates the timely rework of substandard shoe upper semi-finished products, thereby improving the overall shoe production yield.

[0127] Example 2:

[0128] This embodiment discloses a shoe upper last quality inspection system, used to implement the shoe upper last quality inspection method in Embodiment 1; such as Figure 8 As shown, the shoe upper last quality inspection system includes:

[0129] The scanning data acquisition module is used to acquire the scanning data of the upper of the shoe to be tested;

[0130] The location information calculation module is communicatively connected to the scanning data acquisition module. It is used to extract a template shoe upper dataset that matches the shoe upper to be tested from a preset shoe upper template database, and to obtain the location information of the test feature points on the shoe upper to be tested based on the shoe upper scanning data and the template shoe upper dataset. The template shoe upper dataset includes template feature values.

[0131] The feature value calculation module is communicatively connected to the scanning data acquisition module and is used to obtain the detection feature value corresponding to the feature point to be tested based on the location information of the feature point to be tested.

[0132] The detection result acquisition module is communicatively connected to the feature value calculation module and is used to obtain the quality detection result of the shoe upper to be tested based on the detection feature value and the template feature value.

[0133] It should be noted that the working process, working details and technical effects of the shoe upper last quality inspection system provided in this embodiment 2 can be found in embodiment 1, and will not be repeated here.

[0134] Example 3:

[0135] Based on Embodiment 1 or 2, this embodiment discloses an electronic device, which may be a smartphone, tablet computer, laptop computer, or desktop computer, etc. The electronic device may be referred to as a user terminal, portable terminal, desktop terminal, etc., and includes:

[0136] Memory, used to store computer program instructions; and,

[0137] A processor is used to execute the computer program instructions to perform the operation of the shoe upper last quality inspection method as described in any of Embodiment 1.

[0138] Example 4:

[0139] Based on any one of the embodiments 1 to 3, this embodiment discloses a computer program product, including a computer program or instructions, which, when executed by a computer, implements the shoe upper last quality inspection method as described in any one of Embodiment 1.

[0140] Obviously, those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device, or fabricating them separately as individual integrated circuit modules, or fabricating multiple modules or steps as a single integrated circuit module. Thus, the present invention is not limited to any particular hardware and software combination.

[0141] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. These modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for quality inspection of shoe upper lasting, characterized in that: include: Obtain the upper scan data of the shoe upper to be tested; Extract template shoe upper datasets that match the shoe upper to be tested from a preset shoe upper template database, and obtain the location information of the test feature points on the shoe upper to be tested based on the shoe upper scanning data and the template shoe upper dataset; wherein, the template shoe upper dataset includes template feature values; Based on the location information of the feature point to be tested, the detection feature value corresponding to the feature point to be tested is obtained; Based on the detection feature value and the template feature value, the quality inspection result of the shoe upper to be tested is obtained; The shoe upper scanning data is obtained by scanning the outer surface of the shoe upper by circling the shoe upper with a path around it. The shoe upper scanning data includes three-dimensional point cloud data and two-dimensional image data of the shoe upper. Correspondingly, the template shoe upper dataset includes template shoe upper scanning data and template feature point location information. The template shoe upper scanning data includes 3D point cloud template data and 2D image template data. Based on the shoe upper scanning data and the template shoe upper dataset, the location information of the feature points to be tested on the shoe upper to be tested is obtained, including: Point cloud matching is performed on the 3D point cloud data and the 3D point cloud template data to obtain the transformed 3D point cloud data with the smallest error between them and the 3D point cloud template data; wherein, the transformed 3D point cloud data is obtained by performing point cloud transformation on the 3D point cloud data; Based on the 3D point cloud template data and the 2D image template data, obtain the template feature descriptor of the template feature point corresponding to the template feature point position information; Using the template feature point position information of any template feature point as the center of the sphere and with r The search area is defined as the region within a sphere with radius , and the feature descriptors of all test points within the search area of ​​the transformed 3D point cloud data are obtained; wherein, r >0; The feature descriptors of each test point are compared with the template feature descriptors in turn, and the test point with the smallest error with the template feature descriptors is selected as the test feature point of the shoe upper. The three-dimensional coordinate data of the feature point to be tested is extracted from the transformed three-dimensional point cloud data, and the three-dimensional coordinate data is used as the position information of the feature point to be tested on the shoe upper to be tested. Based on the 3D point cloud template data and the 2D image template data, obtain template feature descriptors for the template feature points corresponding to the template feature point location information, including: Based on the 3D point cloud template data and the 2D image template data, obtain the nearest neighbors to the template feature points. k Each sample point, and k The three-dimensional coordinate data and color data of each sample point; among which... k It is a natural number greater than 0; right k The point set consisting of sample points and template feature points is paired up in pairs to obtain... Pairs; Based on the 3D point cloud template data, construct the local map for each pair of points. uvw Coordinate system; the local coordinate system of any point pair uvw The coordinate values ​​in the coordinate system are as follows: , , In the formula, and These are the three-dimensional coordinate data of the two points in the current point pair; For three-dimensional coordinate data The normal vector of the corresponding point; × indicates the cross product sign; Use each part uvw The coordinate values ​​in the coordinate system yield the feature description vectors of each point in the corresponding point pair; where, compared with the three-dimensional coordinate data... The description parameters in the feature description vector of the corresponding point are: ； ； ； ； ； In the formula, , , , and Both refer to descriptive parameters. , and This represents the angular transformation relationship between the coordinate system formed by the line connecting the two points and the base coordinate system. For the local uvw The Euclidean distance between two points at the midpoint of a coordinate system. For the local uvw The Euclidean distance between the color data of two points in a coordinate system corresponding to a point pair; Indicates the inner product symbol; For the local uvw The actual physical distance between two points at the center of the coordinate system; For the local uvw The color difference between two points at the midpoint of the corresponding point in the coordinate system; The template feature points are encoded based on the feature description vectors of all points in the point set to obtain the template feature descriptors of the template feature points; Point cloud matching is performed on the 3D point cloud data and the 3D point cloud template data, including: Obtain the optimal transformation matrix between the 3D point cloud data and the 3D point cloud template data; The three-dimensional point cloud data is transformed according to the optimal transformation matrix to obtain the transformed three-dimensional point cloud data; wherein, the error between the transformed three-dimensional point cloud data and the three-dimensional point cloud template data is minimized. The optimal transformation matrix includes an optimal rotation matrix and an optimal translation matrix; correspondingly, the error between the transformed 3D point cloud data and the 3D point cloud template data is: ； In the formula, Refers to the transformed 3D point cloud data; Refers to the aforementioned three-dimensional point cloud template data; Refers to the optimal rotation matrix; Refers to the optimal translation matrix; Refers to the first in the shoe upper scan data Three-dimensional coordinate data of each point; Refers to the first in the template shoe upper scanning data Three-dimensional coordinate template data of each point; The total number of the three-dimensional point cloud data and the three-dimensional point cloud template data.

2. The method for quality inspection of shoe upper lasting according to claim 1, characterized in that: The template shoe upper dataset also includes basic parameters of the template shoe upper; correspondingly, a template shoe upper dataset matching the shoe upper to be tested is extracted from a preset shoe upper template database, including: Obtain the basic parameters of the shoe upper to be tested; Based on the basic parameters of the upper of the shoe to be tested, a template upper dataset with the same basic parameters as the upper of the shoe to be tested is retrieved from the template upper dataset, and this template upper dataset is used as the template upper dataset that matches the upper of the shoe to be tested.

3. The method for quality inspection of shoe upper lasting according to claim 1, characterized in that: Multiple feature points are set to be tested, and the detection feature value corresponding to the feature point to be tested includes the distance between two specified feature points to be tested. Correspondingly, the distance between two specified feature points to be measured is obtained, including: Obtain the position information of two specified feature points to be tested corresponding to two specified feature points to be tested, and construct a tangent plane based on the position information of the two specified feature points to be tested; wherein, the tangent plane is a plane formed by the straight line connecting the two specified feature points to be tested and the normal vectors corresponding to the two specified feature points to be tested; The intersection of the cutting plane with the three-dimensional point cloud data of the shoe upper to be tested yields the intersection line segment; The intersecting line segments are discretized to obtain multiple discretized line segments; Obtain the lengths of multiple discrete line segments, and sum the lengths of the multiple discrete line segments to obtain the distance between two specified feature points to be measured.

4. The method for quality inspection of shoe upper lasting according to claim 1, characterized in that: If the quality inspection result of the shoe upper to be tested is unqualified, the unqualified location information is output to the preset shoe upper marking device so that the shoe upper marking device can mark the position on the shoe upper to be tested corresponding to the unqualified location information.

5. A shoe upper last quality inspection system, characterized in that: The system is used to implement the shoe upper last quality inspection method as described in any one of claims 1 to 3; the shoe upper last quality inspection system includes: The scanning data acquisition module is used to acquire the scanning data of the upper of the shoe to be tested; The location information calculation module is communicatively connected to the scanning data acquisition module. It is used to extract a template shoe upper dataset that matches the shoe upper to be tested from a preset shoe upper template database, and to obtain the location information of the test feature points on the shoe upper to be tested based on the shoe upper scanning data and the template shoe upper dataset. The template shoe upper dataset includes template feature values. The feature value calculation module is communicatively connected to the scanning data acquisition module and is used to obtain the detection feature value corresponding to the feature point to be tested based on the location information of the feature point to be tested. The detection result acquisition module is communicatively connected to the feature value calculation module and is used to obtain the quality detection result of the shoe upper to be tested based on the detection feature value and the template feature value.

6. An electronic device, characterized in that: include: Memory is used to store computer program instructions; as well as, A processor is configured to execute the computer program instructions to perform the operation of the shoe upper last quality inspection method as described in any one of claims 1 to 3.

7. A computer program product, comprising a computer program or instructions, characterized in that: When the computer program or the instructions are executed by the computer, they implement the shoe upper last quality inspection method as described in any one of claims 1 to 3.

Images