Method, device, equipment and storage medium for gluing objects

Abstract

The present invention discloses a method, device, equipment and storage medium for gluing an object, and relates to the field of gluing an object. The method comprises: obtaining an actual point cloud of the object to be glued; determining a target rotation matrix for the template object based on the actual point cloud and the template point cloud of the template object; determining target gluing points based on the target rotation matrix of the template object and the taught gluing points for the template object, and determining a target gluing posture of a robotic arm for the object to be glued based on the target gluing points; sending the target gluing posture to the robotic arm so that the robotic arm glues the object to be glued based on the target gluing posture. The technical solution of the embodiment of the present invention determines the target gluing points based on the target rotation matrix and the taught gluing points, and then determines the target gluing posture, thereby improving the efficiency and accuracy of object gluing.

Description

Technical Field

[0001] The present invention relates to the technical field of object gluing, and in particular to an object gluing method, device, equipment and storage medium. Background Art

[0002] Gluing is a crucial step in the production of related products, requiring the application of treatment agents, glue coating, and other production processes. The quality of the glue coating is crucial to the quality of the product. Therefore, gluing can be considered a critical process that requires a lot of labor and is time-consuming.

[0003] Current methods for gluing objects are typically based on point cloud matching algorithms, which directly determine the gluing trajectory based on the object's point cloud data. This is overly complex and cannot guarantee matching accuracy, resulting in low accuracy in gluing objects. Therefore, improvements are urgently needed. Summary of the Invention

[0004] The present invention provides a method, device, equipment and storage medium for gluing an object, so as to improve the efficiency and accuracy of gluing an object.

[0005] According to one aspect of the present invention, a method for gluing an object is provided, comprising:

[0006] Obtain the actual point cloud of the object to be glued;

[0007] Determine the target rotation matrix of the template object based on the actual point cloud and the template point cloud of the template object;

[0008] According to the target rotation matrix of the template object and the taught gluing point position of the template object, the target gluing point position is determined, and according to the target gluing point position, the target gluing posture of the robot arm to the gluing object is determined;

[0009] The target gluing posture is sent to the robotic arm, so that the robotic arm glues the object to be glued based on the target gluing posture.

[0010] According to another aspect of the present invention, there is provided an object gluing device, comprising:

[0011] The actual point cloud acquisition module is used to obtain the actual point cloud of the object to be glued;

[0012] A rotation matrix determination module determines a target rotation matrix for the template object based on the actual point cloud and the template point cloud of the template object;

[0013] The gluing point determination module is used to determine the target gluing point according to the target rotation matrix of the template object and the taught gluing point of the template object, and determine the target gluing posture of the robot arm for the object to be glued according to the target gluing point;

[0014] The posture sending module is used to send the target gluing posture to the robotic arm, so that the robotic arm can apply glue to the object to be glued based on the target gluing posture.

[0015] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0016] at least one processor; and

[0017] a memory communicatively connected to at least one processor; wherein,

[0018] The memory stores a computer program that can be executed by at least one processor. The computer program is executed by the at least one processor so that the at least one processor can execute the object gluing method according to any embodiment of the present invention.

[0019] According to another aspect of the present invention, a computer-readable storage medium is provided. The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the object gluing method according to any embodiment of the present invention when executed.

[0020] The technical solution of the embodiment of the present invention obtains the actual point cloud of the object to be glued; determines the target rotation matrix of the template object based on the actual point cloud and the template point cloud of the template object; determines the target gluing point position based on the target rotation matrix of the template object and the taught gluing point position of the template object, and determines the target gluing posture of the robotic arm for the object to be glued based on the target gluing point position; and sends the target gluing posture to the robotic arm so that the robotic arm glues the object to be glued based on the target gluing posture. The technical solution of the embodiment of the present invention determines the target gluing point position based on the target rotation matrix and the taught gluing point position, and then determines the target gluing posture, thereby improving the efficiency and accuracy of object gluing.

[0021] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0023] Figure 1 This is a flow chart of a method for gluing an object provided according to the first embodiment of the present invention;

[0024] Figure 2This is a flow chart of a method for gluing an object provided according to a second embodiment of the present invention;

[0025] Figure 3 This is a flow chart of a method for gluing an object provided according to a third embodiment of the present invention;

[0026] Figure 4 This is a flow chart of a method for gluing an object provided according to a fourth embodiment of the present invention;

[0027] Figure 5 2 is a schematic structural diagram of an object gluing device provided according to a fifth embodiment of the present invention;

[0028] Figure 6 It is a structural schematic diagram of an electronic device for implementing the object gluing method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0029] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0030] It should be noted that the terms "first", "second", "third" and "target" in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0031] In addition, it should be noted that the collection, storage, use, processing, transmission, provision and disclosure of actual point clouds and template point clouds involved in the technical solution of the present invention are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0032] Example 1

[0033] Figure 1A flowchart of a method for gluing an object is provided for embodiment 1 of the present invention. This embodiment is applicable to situations where gluing is performed on an object to be glued. The method can be performed by an object gluing device, which can be implemented in the form of hardware and / or software. The object gluing device can be configured in an electronic device, such as an object gluing main control device.

[0034] like Figure 1 As shown, the method includes:

[0035] S101, obtaining the actual point cloud of the object to be glued.

[0036] In this embodiment, the object to be glued may be an object that currently needs to be glued, such as a sole of a shoe; the actual point cloud may be a point cloud on the surface of the object to be glued, such as a point cloud on a sole.

[0037] In a specific embodiment, the point cloud of the object to be glued can be obtained by shooting with an RGB-D (Red Green Blue-Depth) camera.

[0038] S102 : Determine a target rotation matrix for the template object based on the actual point cloud and the template point cloud of the template object.

[0039] In this embodiment, the template object can be an object used as a template for applying glue, such as the sole of a template shoe; the template point cloud can be a point cloud on the surface of the template object, such as the point cloud of the template sole. The target rotation matrix can be a matrix obtained by rotating the points on the template object. The dimensions of the target rotation matrix are the same as the dimensions of the template point cloud. For example, if the template point cloud is a two-dimensional point cloud, the target rotation matrix is ​​a two-dimensional rotation matrix; if the template point cloud is a three-dimensional point cloud, the target rotation matrix is ​​a three-dimensional matrix.

[0040] Specifically, an object of the same type as the object to be glued can be arbitrarily selected as a template object; the surface point cloud of the template object is obtained as the template point cloud; based on the local features of the actual point cloud, the rotation matrix of the actual point cloud and the transformation matrix of the actual point cloud are determined; based on the local features of the template point cloud, the rotation matrix of the template point cloud and the transformation matrix of the template point cloud are determined; based on the distance between the points in the actual point cloud transformed by the transformation matrix of the actual point cloud and the corresponding points in the template point cloud transformed by the transformation matrix of the template point cloud, the target rotation matrix of the template object is determined.

[0041] It should be noted that the present invention does not limit the local features. For example, the local features may be SHOT (Signature of Histogram of Orientation) features.

[0042] S103, determining the target gluing point according to the target rotation matrix of the template object and the taught gluing point of the template object, and determining the target gluing posture of the robot arm for the object to be glued according to the target gluing point.

[0043] In this embodiment, the teaching gluing point can be a recorded manually determined gluing point; the target gluing point can be a point where the object to be glued is glued; the target gluing posture can be a posture of the robot arm used to glue the object to be glued.

[0044] Specifically, a technician can drag the robotic arm to apply glue to the template object based on the gluing process, using the gluing points during the gluing process as the teaching gluing points. The teaching gluing points are transformed using the target rotation matrix to obtain the target gluing points. Based on the coordinates of the target gluing points in the robotic arm's working coordinate system, the robotic arm's target gluing pose for the object being glued is determined. Using this technical solution, technicians can determine the teaching gluing points based on the gluing process, obtaining relatively accurate teaching gluing points in a single step that better matches the gluing process. This significantly simplifies the complexity of traditional teaching and improves the efficiency and accuracy of gluing objects.

[0045] S104: Send the target gluing posture to the robotic arm, so that the robotic arm applies glue to the object to be glued based on the target gluing posture.

[0046] Specifically, the target gluing posture is sent to the robotic arm via a communication method; accordingly, after receiving the template gluing posture, the robotic arm can apply glue to the object to be glued based on the target gluing posture. It should be noted that the communication method can adopt at least one existing technology, and the present invention is not limited to this.

[0047] The embodiment of the present invention obtains the actual point cloud of the object to be glued; determines the target rotation matrix of the template object based on the actual point cloud and the template point cloud of the template object; determines the target gluing point position based on the target rotation matrix of the template object and the taught gluing point position of the template object, and determines the target gluing posture of the robotic arm for the object to be glued based on the target gluing point position; and sends the target gluing posture to the robotic arm so that the robotic arm glues the object to be glued based on the target gluing posture. Using the above technical solution, the target gluing point position is determined based on the target rotation matrix and the taught gluing point position, and then the target gluing posture is determined, thereby improving the efficiency and accuracy of object gluing.

[0048] Example 2

[0049] Figure 2 This is a flow chart of a method for gluing an object provided by the second embodiment of the present invention. Based on the above embodiments, this embodiment optimizes and improves the operation of determining the target rotation matrix of the template object.

[0050] Furthermore, “determining the target rotation matrix of the template object based on the actual point cloud and the template point cloud of the template object” is refined into “respectively determining the first fast point feature histogram FPFH feature of the actual point cloud and the second FPFH feature of the template point cloud; selecting at least one group of first point sets from the actual point cloud, and selecting at least one group of second point sets from the template point cloud; determining the first rotation matrix and the first transformation matrix of the actual point cloud based on the first FPFH feature and at least one group of first point sets, and determining the second rotation matrix and the second transformation matrix of the template point cloud based on the second FPFH feature and at least one group of second point sets; determining a new actual point cloud based on the first transformation matrix and the actual point cloud, and determining a new template point cloud based on the second transformation matrix and the template point cloud; comparing the distance between the new actual point cloud and the new template point cloud, and determining the target rotation matrix of the template object based on the comparison result”, so as to improve the operation of determining the target rotation matrix of the template object.

[0051] It should be noted that for the parts not described in detail in the embodiments of the present invention, reference can be made to the description of the aforementioned embodiments.

[0052] like Figure 2 As shown, the method includes:

[0053] S201: Acquire the actual point cloud of the object to be glued.

[0054] S202 : Determine first FPFH (Fast Point Feature Histograms) features of the actual point cloud and second FPFH features of the template point cloud respectively.

[0055] The first FPFH feature may be the FPFH feature of the point in the actual point cloud; the second FPFH feature may be the FPFH feature of the point in the template point cloud.

[0056] In this embodiment, a first FPFH feature corresponding to each point in the actual point cloud is determined; and a second FPFH feature corresponding to each point in the template point cloud is determined.

[0057] Taking a point in an actual point cloud as an example, the determination of the first FPFH feature of the point is explained as follows: the number of points in the neighborhood of the point in the actual point cloud is determined, and the distance between each point in the neighborhood of the point and the point is determined; the SPFH (Simplified Point Feature Histogram) feature of each point in the neighborhood of the point is determined; the ratio of the SPFH feature of each point in the neighborhood to the distance between each corresponding point and the point is determined; the weighted average result of each ratio is determined, that is, all the ratios are accumulated and divided by the number of points in the neighborhood; the addition result of the weighted average result and the simplified point feature histogram SPFH feature of the point is determined, and the addition result is used as the first FPFH feature of the point.

[0058] For example, the first FPFH feature of a point in the actual point cloud can be determined according to the following formula:

[0059]

[0060] Among them, S q represents the q point in the actual point cloud S; S i Represents point i in the actual point cloud S; FPFH(S q ) represents point S q FPFH characteristics; SPFH (S q ) represents point S q SPFH characteristics; SPFH(S i ) represents point S i Simplified point feature histogram SPFH feature; k represents point S q The number of points in the neighborhood of ; ω i Indicates point S q With point S i The distance between them.

[0061] Correspondingly, taking a point in the template point cloud as an example, the process of determining the second FPFH feature of the point is similar to the above-mentioned process of determining the first FPFH feature of the point in the actual point cloud, and will not be repeated here.

[0062] S203 : Select at least one first point set from the actual point cloud, and select at least one second point set from the template point cloud.

[0063] In this embodiment, the first point set refers to a set of points selected from the actual point cloud, and may include at least one point. The second point set refers to a set of points selected from the template point cloud, and may include at least one point.

[0064] Optionally, a first number of points are randomly sampled from the actual point cloud as a first point set, and at least one group of the first point set is randomly sampled. Correspondingly, a first number of points are randomly sampled from the template point cloud as a second point set, and at least one group of the second point set is randomly sampled. For example, N groups of first point sets are selected from the actual point cloud, each group of first point sets includes M points; and N groups of first point sets are selected from the template point cloud, each group of first point sets includes M points; where N and M are both natural numbers greater than or equal to 1. The first number is less than or equal to a sampling point threshold. The sampling point threshold can be independently set by technicians based on actual needs and practical experience, and is not limited in this invention.

[0065] It should be noted that the first distance between the two points in the first point set is greater than the distance threshold, and the second distance between the two points in the second point set is greater than the distance threshold. The distance threshold can be independently set by technicians based on actual needs and practical experience, and the present invention does not limit this.

[0066] Specifically, after selecting the first point set, the distance between every two points in the first point set is determined as the first distance. If the first distance between every two points is greater than the distance threshold, the first point set is retained; otherwise, the first point set is discarded, and a new first point set is selected from the actual point cloud, and a new first distance between the two points in the new first point set is determined until the first distance between every two points in the first point set is greater than the distance threshold.

[0067] Correspondingly, after selecting the second point set, the distance between every two points in the second point set is determined as the second distance. If the second distance between every two points is greater than the distance threshold, the second point set is retained; otherwise, the second point set is discarded, and a new second point set is selected from the actual point cloud, and a new second distance between the two points in the new second point set is determined until the second distance between every two points in the second point set is greater than the distance threshold.

[0068] It can be understood that by adopting the above technical solution, the first distance between two points in the first point set is greater than the distance threshold, and the second distance between two points in the second point set is greater than the distance threshold. This can avoid the situation where the covariance matrix accuracy of the actual point cloud and / or the covariance matrix accuracy of the template point cloud are poor due to the point distance in the first point set and / or the second point set being too close, which in turn leads to the poor accuracy of the first rotation matrix, the second rotation matrix, the first transformation matrix and / or the second transformation matrix. This lays the foundation for improving the accuracy of the target rotation matrix and ensuring the accuracy of the object gluing.

[0069] S204. Determine a first rotation matrix and a first transformation matrix of the actual point cloud based on the first FPFH feature and at least one set of first points, and determine a second rotation matrix and a second transformation matrix of the template point cloud based on the second FPFH feature and at least one set of second points.

[0070] The first rotation matrix may be a rotation matrix that rotates the coordinates of each point in the actual point cloud; the first transformation matrix may be a transformation matrix that transforms the coordinates of each point in the actual point cloud. The first rotation matrix and the first transformation matrix have the same dimensions as the actual point cloud. For example, if the actual point cloud is a three-dimensional point cloud, the first rotation matrix and the first transformation matrix are three-dimensional matrices. The second rotation matrix may be a rotation matrix that rotates the coordinates of each point in the template point cloud; the second transformation matrix may be a transformation matrix that transforms the coordinates of each point in the template point cloud. The second rotation matrix and the second transformation matrix have the same dimensions as the template point cloud. For example, if the template point cloud is a three-dimensional point cloud, the second rotation matrix and the second transformation matrix are three-dimensional matrices.

[0071] Optionally, based on a certain algorithm, the first rotation matrix and the first transformation matrix of the actual point cloud can be determined according to the first FPFH feature and at least one set of first points; at the same time, the same algorithm can be used to determine the second rotation matrix and the second transformation matrix of the template point cloud according to the second FPFH feature and at least one set of second points.

[0072] S205 . Determine a new actual point cloud according to the first transformation matrix and the actual point cloud, and determine a new template point cloud according to the second transformation matrix and the template point cloud.

[0073] The new actual point cloud may be an actual point cloud obtained by transforming each point in the actual point cloud by the first transformation matrix; and the new template point cloud may be a template point cloud obtained by transforming each point in the template point cloud by the second transformation matrix.

[0074] Optionally, the first transformation matrix is ​​multiplied by the actual point cloud to obtain a new actual point cloud, and the second transformation matrix is ​​multiplied by the template point cloud to obtain a new template point cloud.

[0075] It can be understood that by adopting the above technical solution, by multiplying the first transformation matrix with the actual point cloud to obtain a new actual point cloud, and multiplying the second transformation matrix with the template point cloud to obtain a new template point cloud, the actual point cloud and the template point cloud in the working coordinate system of the robotic arm can be obtained, so as to facilitate the comparison of the distance between the points of the actual point cloud in the working coordinate system of the robotic arm and the points of the template point cloud.

[0076] S206 : Compare the distance between the new actual point cloud and the new template point cloud, and determine the target rotation matrix for the template object based on the comparison result.

[0077] Optionally, determine the third distance between a point in the new actual point cloud and the point in the new template point cloud corresponding to the point; if each third distance is within the error range, use the second rotation matrix as the target rotation matrix; otherwise, reselect at least one set of first points from the actual point cloud, and reselect at least one set of second points from the template point cloud.

[0078] The third distance may be the distance between a point in the new actual point cloud and a point in the new template point cloud corresponding to the point.

[0079] Specifically, if the third distance between each point in the new actual point cloud and the point in the new template point cloud corresponding to each point is within the error range, the second rotation matrix is ​​used as the target rotation matrix; otherwise, at least one set of first point sets is reselected from the actual point cloud, and at least one set of second point sets is reselected from the template point cloud; based on the first FPFH feature and the reselected at least one set of first point sets, the new first rotation matrix and the new first transformation matrix of the actual point cloud are determined, and based on the second FPFH feature and the reselected at least one set of second point sets, the new second rotation matrix and the new second transformation matrix of the template point cloud are determined; based on the new first transformation matrix and the actual point cloud, the new actual point cloud is determined, and based on the new second transformation matrix and the template point cloud, the new template point cloud is determined; the third distance between the point in the new actual point cloud and the point in the new template point cloud corresponding to the point is determined; until the third distance between the point in the new actual point cloud and the point in the new template point cloud corresponding to the point is within the error range.

[0080] It should be noted that the error range can be independently set by technical personnel based on actual needs and practical experience, and the present invention does not limit this.

[0081] In a specific embodiment, the error range can be 5 mm, that is, if the third distance between each point in the new actual point cloud and the point in the new template point cloud corresponding to each point is within 5 mm, the second rotation matrix is ​​used as the target rotation matrix; otherwise, at least one set of first points is reselected from the actual point cloud, and at least one set of second points is reselected from the template point cloud.

[0082] It can be understood that, by adopting the above technical solution, if the third distances between each point in the new actual point cloud and the points in the new template point cloud corresponding to each point are within the error range, the second rotation matrix is ​​used as the target rotation matrix; otherwise, at least one set of first points is reselected from the actual point cloud, and at least one set of second points is reselected from the template point cloud, thereby improving the degree of similarity between the points in the new actual point cloud and the corresponding points in the new template point cloud, thereby improving the accuracy of object gluing.

[0083] S207 , determining target gluing points according to the target rotation matrix of the template object and the taught gluing points of the template object, and determining the target gluing posture of the robot arm for the object to be glued according to the target gluing points.

[0084] S208 , sending the target gluing posture to the robotic arm, so that the robotic arm applies glue to the object to be glued based on the target gluing posture.

[0085] The embodiment of the present invention determines the first FPFH feature of the actual point cloud and the second FPFH feature of the template point cloud respectively; selects at least one first point set from the actual point cloud and at least one second point set from the template point cloud; determines the first rotation matrix and the first transformation matrix of the actual point cloud based on the first FPFH feature and the at least one first point set, and determines the second rotation matrix and the second transformation matrix of the template point cloud based on the second FPFH feature and the at least one second point set; determines a new actual point cloud based on the first transformation matrix and the actual point cloud, and determines a new template point cloud based on the second transformation matrix and the template point cloud; compares the distance between the new actual point cloud and the new template point cloud, and determines the target rotation matrix for the template object based on the comparison result. The above scheme can improve the matching degree between the target gluing point position obtained after the target rotation matrix converts the teaching gluing point position and the object to be glued, thereby improving the matching degree between the target gluing posture and the object to be glued, and improving the accuracy of gluing the object.

[0086] Example 3

[0087] Figure 3 This is a flowchart of a method for gluing an object provided by Example 3 of the present invention. Based on the above embodiments, this embodiment optimizes and improves the determination operations of the first rotation matrix, the first transformation matrix, the second rotation matrix and the second transformation matrix.

[0088] It should be noted that for the parts not described in detail in the embodiments of the present invention, reference can be made to the description of the aforementioned embodiments.

[0089] like Figure 3 As shown, the method includes:

[0090] S301: Acquire the actual point cloud of the object to be glued.

[0091] S302 : Determine first FPFH features of the actual point cloud and second FPFH features of the template point cloud respectively.

[0092] S303 : Select at least one first point set from the actual point cloud, and select at least one second point set from the template point cloud.

[0093] S304. Determine a first rotation matrix and a first transformation matrix of the actual point cloud based on the first FPFH feature and at least one first point set, and determine a second rotation matrix and a second transformation matrix of the template point cloud based on the second FPFH feature and at least one second point set.

[0094] Optionally, the first rotation matrix and the first transformation matrix of the actual point cloud are determined according to the first FPFH feature and at least one group of first point sets, including: determining the first Euclidean distance between every two points in the first point set according to the first FPFH feature, and determining the first center of mass from the first point set according to the first Euclidean distance; determining the covariance matrix of the actual point cloud according to the at least one group of first point sets and the first center of mass of at least one group of first point sets; determining the first rotation matrix and the first transformation matrix of the actual point cloud according to the covariance matrix of the actual point cloud; accordingly, the second rotation matrix and the second transformation matrix of the template point cloud are determined according to the second FPFH feature and at least one group of second point sets, including: determining the second Euclidean distance between every two points in the second point set according to the second FPFH feature, and determining the second center of mass from the second point set according to the second Euclidean distance; determining the covariance matrix of the template point cloud according to the covariance matrix of the template point cloud.

[0095] Among them, the FPFH feature can be represented as a 32-bit array; the centroid can be the point with the smallest sum of Euclidean distances to other points in the point set.

[0096] Taking a group of first point sets as an example, the method for determining the first centroid of the first point set is explained: subtract the numerical value of the corresponding bit in the 32-bit array of the FPFH feature of a point in the first point set from the 32-bit array of the FPFH feature of another point in the first point set to obtain a 32-bit subtraction result; square each subtraction result in the subtraction result, and superimpose the 32-bit subtraction results to obtain the first Euclidean distance between the two points; determine the first Euclidean distance between the point and the other points in the first point set by the above method for determining the first Euclidean distance between the two points; superimpose the first Euclidean distance between the point and the other points in the first point set to obtain the total Euclidean distance of the point; determine the total Euclidean distance of each point in the first point set by the above method for determining the total Euclidean distance of one point; compare the total Euclidean distances of each point, and take the point in the first point set corresponding to the smallest total Euclidean distance as the first centroid of the group of first point sets.

[0097] Determine the first centroid of each group of first point sets by the above-mentioned first centroid method, and remove the first centroid from each group of first point sets; arrange the first point sets of each group after removing the first centroid in rows to form a first point set matrix, and the element values ​​of the matrix are the coordinate values ​​of each point in the first point set; determine the average value of each column element in the first point set matrix, and subtract the average value of the corresponding column from the element value of each column of the first point set matrix to obtain a new first point set matrix; determine the first point set transpose matrix corresponding to the new first point set matrix; multiply the new first point set matrix by the corresponding first point set transpose matrix, and divide it by the number of groups of the first point set to obtain the covariance matrix of the actual point cloud. For example, the covariance matrix of the actual point cloud can be obtained by the following formula:

[0098]

[0099] Among them, COV S Represents the covariance matrix of the actual point cloud S, m S represents the number of groups of the first point set; X S represents the new first point set matrix; X S T Represents the first point set transposed matrix corresponding to the new first point set matrix.

[0100] Determine the two first eigenvalues ​​of the actual point cloud covariance matrix, and use the vector whose multiplication result with the corresponding first eigenvalue is equal to the multiplication result with the actual point cloud covariance matrix as the first eigenvector of the covariance matrix of the actual point cloud. For example, the first eigenvector of the covariance matrix of the actual point cloud can be obtained by the following formula:

[0101] COV S v1=λ1v1;

[0102] COV S v2=λ2v2;

[0103] Among them, COV S is the covariance matrix of the actual point cloud; v1 is a first eigenvector; v2 is another first eigenvector; λ1 is the eigenvalue corresponding to the first eigenvector v1; λ2 is the eigenvalue corresponding to the first eigenvector v2. It should be noted that λ1 and λ2 can be determined using at least one existing technique, and the present invention is not limited thereto.

[0104] The two first eigenvectors are cross-producted to obtain a first cross-product vector; the two first eigenvectors are combined with the first cross-product vector to obtain a first rotation matrix; the first centroid of the first point set is combined with the first rotation matrix to obtain a first transformation matrix.

[0105] Correspondingly, the second rotation matrix and the second transformation matrix of the template point cloud are determined in the same manner as the first rotation matrix and the first transformation matrix of the actual point cloud, and are not described in detail here.

[0106] It can be understood that, by adopting the above technical solution, by determining the first centroid of at least one group of first point sets, and determining the covariance matrix of the actual point cloud based on the at least one group of first point sets; determining the first rotation matrix and the first transformation matrix of the actual point cloud based on the covariance matrix of the actual point cloud; by determining the second centroid of at least one group of second point sets, and determining the covariance matrix of the template point cloud based on the at least one group of second point sets; determining the second rotation matrix and the second transformation matrix of the template point cloud based on the covariance matrix of the template point cloud, the accuracy of the first rotation matrix and the first transformation matrix of the actual point cloud is improved, the accuracy of the second rotation matrix and the second transformation matrix of the template point cloud is improved, thereby improving the accuracy of the target rotation matrix and improving the accuracy of the object gluing.

[0107] S305 : Determine a new actual point cloud according to the first transformation matrix and the actual point cloud, and determine a new template point cloud according to the second transformation matrix and the template point cloud.

[0108] S306 : Compare the distance between the new actual point cloud and the new template point cloud, and determine the target rotation matrix for the template object based on the comparison result.

[0109] S307 , determining target gluing points according to the target rotation matrix of the template object and the taught gluing points of the template object, and determining the target gluing posture of the robot arm for the object to be glued according to the target gluing points.

[0110] S308 , sending the target gluing posture to the robotic arm, so that the robotic arm applies glue to the object to be glued based on the target gluing posture.

[0111] The technical solution of the embodiment of the present invention is to determine the first Euclidean distance between every two points in the first point set according to the first FPFH feature, and determine the first center of mass from the first point set according to the first Euclidean distance; determine the covariance matrix of the actual point cloud according to at least one group of first point sets and the first center of mass of at least one group of first point sets; determine the first rotation matrix and the first transformation matrix of the actual point cloud according to the covariance matrix of the actual point cloud; accordingly, determine the second rotation matrix and the second transformation matrix of the template point cloud according to the second FPFH feature and at least one group of second point sets, including: determining the second Euclidean distance between every two points in the second point set according to the second FPFH feature, and determining the second center of mass from the second point set according to the second Euclidean distance; determine the covariance matrix of the template point cloud according to the at least one group of second point sets and the second center of mass of at least one group of second point sets; and determine the second rotation matrix and the second transformation matrix of the template point cloud according to the covariance matrix of the template point cloud. The above technical solution improves the accuracy of the first rotation matrix and the first transformation matrix of the actual point cloud, improves the accuracy of the second rotation matrix and the second transformation matrix of the template point cloud, thereby improving the accuracy of the target rotation matrix and the accuracy of object gluing.

[0112] Example 4

[0113] Figure 4 This is a flow chart of a method for gluing an object provided in the fourth embodiment of the present invention. Based on the above embodiments, this embodiment optimizes and improves the operation of determining the target gluing point.

[0114] Furthermore, “determining the target gluing point position based on the target rotation matrix of the template object and the taught gluing point position of the template object” is refined into “calibrating the target rotation matrix; determining the target gluing point position based on the calibrated target rotation matrix and the taught gluing point position of the template object” to improve the target gluing point determination operation.

[0115] It should be noted that for the parts not described in detail in the embodiments of the present invention, reference can be made to the description of the aforementioned embodiments.

[0116] like Figure 4 As shown, the method includes:

[0117] S401: Acquire the actual point cloud of the object to be glued.

[0118] S402: Determine a target rotation matrix for the template object based on the actual point cloud and the template point cloud of the template object.

[0119] S403: Calibrate the target rotation matrix.

[0120] Specifically, a point cloud registration algorithm is used to calibrate the target rotation matrix to improve the accuracy of the target rotation matrix, thereby improving the accuracy of the target gluing point. It should be noted that the present invention does not limit the point cloud registration algorithm. For example, the point cloud registration algorithm can be an ICP (Iterative Closest Point) algorithm, a NICP (Normal Iterative Closest Point) algorithm, and a PL-ICP (Point to Line-Iterative Closest Point) algorithm.

[0121] S404: Determine target gluing points based on the calibrated target rotation matrix and the taught gluing points of the template object, and determine the target gluing posture of the robot arm for the object to be glued based on the target gluing points.

[0122] Specifically, the taught gluing points are transformed according to the calibrated target rotation matrix to obtain updated target gluing points, the target gluing posture is determined according to the updated target gluing points, and the target gluing points of the robotic arm on the object to be glued are determined according to the target gluing points.

[0123] S405 , sending the target gluing posture to the robotic arm, so that the robotic arm applies glue to the object to be glued based on the target gluing posture.

[0124] This embodiment of the present invention calibrates the target rotation matrix and determines the target gluing points based on the calibrated target rotation matrix and the taught gluing points on the template object. This technical solution improves the accuracy of the target rotation matrix, the accuracy of the target gluing points, and, consequently, the accuracy of gluing the object.

[0125] Example 5

[0126] Figure 5 This is a structural schematic diagram of an object gluing device provided in Example 4 of the present invention. This embodiment can be applied to situations where gluing is performed on objects to be glued. The object gluing device can be implemented in the form of hardware and / or software. The object gluing device can be configured in an electronic device, such as an object gluing main control device.

[0127] like Figure 5 As shown, the device includes: an actual point cloud acquisition module 501, a rotation matrix determination module 502, a glue point determination module 503 and a posture sending module 504.

[0128] The actual point cloud acquisition module 501 is used to acquire the actual point cloud of the object to be glued;

[0129] The rotation matrix determination module 502 determines a target rotation matrix for the template object based on the actual point cloud and the template point cloud of the template object;

[0130] The gluing point determination module 503 is used to determine the target gluing point according to the target rotation matrix of the template object and the taught gluing point of the template object, and determine the target gluing posture of the robot arm on the object to be glued according to the target gluing point;

[0131] The posture sending module 504 is used to send the target gluing posture to the robotic arm, so that the robotic arm can apply glue to the object to be glued based on the target gluing posture.

[0132] In the embodiment of the present invention, the actual point cloud of the object to be glued is obtained by the actual point cloud acquisition module; the rotation matrix determination module determines the target rotation matrix of the template object based on the actual point cloud and the template point cloud of the template object; the gluing point determination module determines the target gluing point according to the target rotation matrix of the template object and the taught gluing point of the template object, and determines the target gluing posture of the robot arm for the object to be glued based on the target gluing point; the posture sending module sends the target gluing posture to the robot arm, so that the robot arm glues the object to be glued based on the target gluing posture. Using the above technical solution,

[0133] Optionally, the rotation matrix determination module 502 includes:

[0134] A feature determination unit, configured to respectively determine a first FPFH feature of the actual point cloud and a second FPFH feature of the template point cloud;

[0135] a point set selection unit, configured to select at least one first point set from the actual point cloud and at least one second point set from the template point cloud;

[0136] a matrix determination unit, configured to determine a first rotation matrix and a first transformation matrix of the actual point cloud based on the first FPFH feature and at least one first point set, and to determine a second rotation matrix and a second transformation matrix of the template point cloud based on the second FPFH feature and at least one second point set;

[0137] a point cloud determination unit, configured to determine a new actual point cloud based on the first transformation matrix and the actual point cloud, and to determine a new template point cloud based on the second transformation matrix and the template point cloud;

[0138] The rotation matrix determination unit is used to compare the distance between the new actual point cloud and the new template point cloud, and determine the target rotation matrix of the template object based on the comparison result.

[0139] Optionally, in the rotation matrix determination module 502 , a first distance between two points in the first point set is greater than a distance threshold, and a second distance between two points in the second point set is greater than the distance threshold.

[0140] Optionally, the matrix determination unit includes:

[0141] a first centroid determining subunit, configured to determine a first Euclidean distance between every two points in the first point set according to the first FPFH feature, and determine a first centroid from the first point set according to the first Euclidean distance;

[0142] an actual covariance matrix determining unit, configured to determine a covariance matrix of an actual point cloud based on at least one first point set and at least one first centroid of the first point set;

[0143] A first rotation matrix determining unit, configured to determine a first rotation matrix and a first transformation matrix of the actual point cloud according to a covariance matrix of the actual point cloud;

[0144] Accordingly, the matrix determination unit includes:

[0145] a second centroid determining subunit, configured to determine a second Euclidean distance between every two points in the second point set according to the second FPFH feature, and determine a second centroid from the second point set according to the second Euclidean distance;

[0146] A template covariance matrix, configured to determine a covariance matrix of the template point cloud based on the at least one set of second point sets and the at least one set of second centroids of the second point sets;

[0147] The second rotation matrix determining subunit is used to determine the second rotation matrix and the second transformation matrix of the template point cloud according to the covariance matrix of the template point cloud.

[0148] Optionally, a point cloud determination unit includes:

[0149] The point cloud determination subunit is used to multiply the first transformation matrix with the actual point cloud to obtain a new actual point cloud, and multiply the second transformation matrix with the template point cloud to obtain a new template point cloud.

[0150] Optionally, the rotation matrix determination unit includes:

[0151] a third distance determining subunit, configured to determine a third distance between a point in a new actual point cloud and a point in a new template point cloud corresponding to the point;

[0152] The rotation matrix determination subunit is used to, if all third distances are within the error range, use the second rotation matrix as the target rotation matrix; otherwise, reselect at least one set of first points from the actual point cloud and reselect at least one set of second points from the template point cloud.

[0153] Optionally, the gluing point determination module 503 includes:

[0154] A matrix calibration unit, used to calibrate the target rotation matrix;

[0155] The gluing point determination unit is used to determine the target gluing point according to the calibrated target rotation matrix and the taught gluing point of the template object.

[0156] The object gluing device provided by the embodiment of the present invention can execute the object gluing method provided by any embodiment of the present invention, and has corresponding functional modules and beneficial effects for executing the object gluing method.

[0157] Example 6

[0158] Figure 6 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0159] like Figure 6 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., which is communicatively connected to the at least one processor 11. The memory stores a computer program that can be executed by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. Various programs and data required for the operation of the electronic device 10 can also be stored in the RAM 13. The processor 11, ROM 12, and RAM 13 are connected to each other via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0160] Multiple components in the electronic device 10 are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0161] The processor 11 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the object gluing method.

[0162] In some embodiments, the object gluing method can be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the object gluing method described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to perform the object gluing method in any other suitable manner (e.g., via firmware).

[0163] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0164] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the computer program is executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer program may be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0165] In the context of the present invention, computer-readable storage media can be tangible media that can contain or store a computer program for use with an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. Computer-readable storage media can include but are not limited to electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, computer-readable storage media can be machine-readable signal media. More specific examples of machine-readable storage media can include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0166] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0167] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0168] A computing system may include clients and servers. The clients and servers are typically remote from each other and typically interact via a communication network. This client-server relationship arises through computer programs running on the respective computers, creating a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host. This server is a hosting product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosting and VPS services.

[0169] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the present invention can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved. This is not limited herein.

[0170] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A method for gluing an object, characterized in that: include: Obtain the actual point cloud of the object to be glued; determining a target rotation matrix for the template object based on the actual point cloud and the template point cloud of the template object; Determining target gluing points according to the target rotation matrix of the template object and the taught gluing points of the template object, and determining the target gluing posture of the robot arm for the object to be glued according to the target gluing points; Sending the target gluing posture to the robotic arm, so that the robotic arm applies glue to the object to be glued based on the target gluing posture; Wherein, determining the target rotation matrix of the template object according to the actual point cloud and the template point cloud of the template object includes: Determining a first FPFH (Fast Point Feature Histogram) feature of the actual point cloud and a second FPFH feature of the template point cloud respectively; Selecting at least one first point set from the actual point cloud and at least one second point set from the template point cloud; Determining a first rotation matrix and a first transformation matrix of the actual point cloud based on the first FPFH feature and the at least one set of first points, and determining a second rotation matrix and a second transformation matrix of the template point cloud based on the second FPFH feature and the at least one set of second points; Determine a new actual point cloud according to the first transformation matrix and the actual point cloud, and determine a new template point cloud according to the second transformation matrix and the template point cloud; The distance between the new actual point cloud and the new template point cloud is compared, and a target rotation matrix for the template object is determined based on the comparison result.

2. The method according to claim 1, characterized in that A first distance between two points in the first point set is greater than a distance threshold, and a second distance between two points in the second point set is greater than a distance threshold.

3. The method according to claim 1, characterized in that Determining a first rotation matrix and a first transformation matrix of the actual point cloud according to the first FPFH feature and the at least one first point set includes: determining a first Euclidean distance between every two points in the first point set based on the first FPFH feature, and determining a first centroid from the first point set based on the first Euclidean distance; Determining a covariance matrix of the actual point cloud based on the at least one set of first point sets and the first centroid of the at least one set of first point sets; Determining a first rotation matrix and a first transformation matrix of the actual point cloud according to the covariance matrix of the actual point cloud; Accordingly, determining a second rotation matrix and a second transformation matrix of the template point cloud according to the second FPFH feature and the at least one second point set includes: determining a second Euclidean distance between every two points in the second point set according to the second FPFH feature, and determining a second centroid from the second point set according to the second Euclidean distance; Determining a covariance matrix of the template point cloud based on the at least one set of second point sets and the second centroid of the at least one set of second point sets; A second rotation matrix and a second transformation matrix of the template point cloud are determined according to the covariance matrix of the template point cloud.

4. The method according to claim 1, wherein The determining a new actual point cloud according to the first transformation matrix and the actual point cloud, and determining a new template point cloud according to the second transformation matrix and the template point cloud, includes: The first transformation matrix is ​​multiplied by the actual point cloud to obtain a new actual point cloud, and the second transformation matrix is ​​multiplied by the template point cloud to obtain a new template point cloud.

5. The method according to claim 1, wherein The comparing the distance between the new actual point cloud and the new template point cloud, and determining the target rotation matrix for the template object according to the comparison result, includes: Determine a third distance between a point in the new actual point cloud and a point in the new template point cloud corresponding to the point; If each of the third distances is within the error range, the second rotation matrix is ​​used as the target rotation matrix; otherwise, at least one set of first points is reselected from the actual point cloud, and at least one set of second points is reselected from the template point cloud.

6. The method according to claim 1, characterized in that The method of determining target gluing points according to the target rotation matrix of the template object and the taught gluing points of the template object further includes: calibrating the target rotation matrix; The target gluing point position is determined according to the calibrated target rotation matrix and the taught gluing point position of the template object.

7. An object gluing device, characterized in that: include: The actual point cloud acquisition module is used to obtain the actual point cloud of the object to be glued; a rotation matrix determination module, which determines a target rotation matrix for the template object based on the actual point cloud and the template point cloud of the template object; a gluing point determination module, configured to determine target gluing points according to a target rotation matrix of the template object and the taught gluing points of the template object, and determine a target gluing posture of the manipulator for the object to be glued according to the target gluing points; A posture sending module, used for sending the target gluing posture to the robotic arm, so that the robotic arm can apply glue to the object to be glued based on the target gluing posture; Wherein, the rotation matrix determination module includes: a feature determination unit, configured to respectively determine a first FPFH (Fast Point Feature Histogram) feature of the actual point cloud and a second FPFH feature of the template point cloud; a point set selection unit, configured to select at least one first point set from the actual point cloud and at least one second point set from the template point cloud; a matrix determination unit, configured to determine a first rotation matrix and a first transformation matrix of the actual point cloud based on the first FPFH feature and the at least one first point set, and to determine a second rotation matrix and a second transformation matrix of the template point cloud based on the second FPFH feature and the at least one second point set; a point cloud determining unit, configured to determine a new actual point cloud based on the first transformation matrix and the actual point cloud, and to determine a new template point cloud based on the second transformation matrix and the template point cloud; The rotation matrix determination unit is used to compare the distance between the new actual point cloud and the new template point cloud, and determine the target rotation matrix for the template object according to the comparison result.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor. The computer program is executed by the at least one processor to enable the at least one processor to perform the object gluing method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the object gluing method according to any one of claims 1 to 6 when executed.

Images