Label pasting method and device, equipment and storage medium

Abstract

The invention provides a label pasting method, device and equipment and a storage medium, and the method comprises the steps: collecting a first image of a label supply device based on first image collection equipment, and collecting a second image of a target workpiece based on second image collection equipment; determining a first position parameter of the target label under the base coordinate system according to the first image, the first image coordinate system and the first conversion relation, and determining a second position parameter of the target workpiece under the base coordinate system according to the second image, the second image coordinate system and the second conversion relation; and based on the first position parameter and the second position parameter, the labeling robot is controlled to paste the target label to the target workpiece. By adopting the method, the label pasting precision is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of computer processing, and in particular to a label sticking method, device, equipment and storage medium. Background Art

[0002] In the manufacturing process of electronic products, it is usually necessary to label the products. Labeling refers to pasting product labels such as product model, production date and brand logo on electronic products. Technical specifications, etc.

[0003] Common labeling methods include using visual sensors to guide robots to label. However, this labeling method has poor labeling accuracy due to differences in the position and angle of the label each time the label stripper removes the label, errors in the position of the workpiece to be labeled, and wear problems after the robot runs at high speed for a long time.

[0004] Therefore, how to improve labeling accuracy has become a technical problem that needs to be solved urgently. Summary of the invention

[0005] The present disclosure provides a label sticking method, device, equipment and storage medium to at least solve the above technical problems existing in the prior art.

[0006] According to a first aspect of the present disclosure, a label pasting method is provided, the method comprising:

[0007] A first image of the label supply device is acquired based on a first image acquisition device, and a second image of the target workpiece is acquired based on a second image acquisition device; wherein the first image acquisition device, the second image acquisition device and the labeling robot are pre-calibrated devices, a first image coordinate system of the first image acquisition device and a base coordinate system of the labeling robot have a first conversion relationship, and a second image coordinate system of the second image acquisition device and the base coordinate system have a second conversion relationship;

[0008] Determine a first position parameter of the target tag in the base coordinate system according to the first image, the first image coordinate system and the first conversion relationship, and determine a second position parameter of the target workpiece in the base coordinate system according to the second image, the second image coordinate system and the second conversion relationship;

[0009] Based on the first position parameter and the second position parameter, the labeling robot is controlled to attach the target label to the target workpiece.

[0010] In one possible implementation manner, the second image coordinate system of the second image acquisition device satisfies a first sub-conversion relationship with the terminal coordinate system of the labeling robot, and the base coordinate system satisfies a second sub-conversion relationship with the terminal coordinate system; the second conversion relationship includes a first sub-conversion relationship and a second sub-conversion relationship;

[0011] Determining the second position parameter of the target workpiece in the base coordinate system according to the second image, the second image coordinate system and the second conversion relationship includes:

[0012] Determine the image position coordinates of the target workpiece in the second image coordinate system according to the second image;

[0013] Based on the first sub-conversion relationship, converting the image position coordinates into first workpiece position coordinates in the terminal coordinate system;

[0014] According to the second sub-conversion relationship, the first workpiece position coordinates are converted into second workpiece position coordinates in the base coordinate system, and the second workpiece position coordinates are used as second position parameters.

[0015] In one possible implementation manner, the second sub-conversion relationship is determined based on each joint angle of the labeling robot.

[0016] In one possible implementation, the first conversion relationship includes a third coordinate conversion matrix satisfied between the second image coordinate system of the second image acquisition device and the terminal coordinate system of the labeling robot;

[0017] The determining, according to the first image, the first image coordinate system and the first conversion relationship, a first position parameter of the target tag in the base coordinate system comprises:

[0018] Determine the image position coordinates of the target tag in the first image coordinate system according to the first image;

[0019] Based on the third coordinate conversion matrix, the image position coordinates are converted into label position coordinates in the base coordinate system, and the label position coordinates are used as the first position parameter.

[0020] In one possible implementation, the controlling the labeling robot to attach the target label to the target workpiece based on the first position parameter and the second position parameter includes:

[0021] Determine the offset distance and offset angle of the end of the labeling robot according to the first position parameter and the second position parameter;

[0022] According to the offset distance and the offset angle, the labeling robot is controlled to attach the target label to the target workpiece.

[0023] According to a second aspect of the present disclosure, a label sticking device is provided, the device comprising:

[0024] An image acquisition module, used for acquiring a first image of the label supply device based on a first image acquisition device, and acquiring a second image of the target workpiece based on a second image acquisition device; wherein the first image acquisition device, the second image acquisition device and the labeling robot are pre-calibrated devices, a first image coordinate system of the first image acquisition device and a base coordinate system of the labeling robot have a first conversion relationship, and a second image coordinate system of the second image acquisition device and the base coordinate system have a second conversion relationship;

[0025] a parameter determination module, configured to determine a first position parameter of the target tag in the base coordinate system according to the first image, the first image coordinate system and the first conversion relationship, and to determine a second position parameter of the target workpiece in the base coordinate system according to the second image, the second image coordinate system and the second conversion relationship;

[0026] A pasting module is used to control the labeling robot to paste the target label onto the target workpiece based on the first position parameter and the second position parameter.

[0027] In one possible implementation manner, the second image coordinate system of the second image acquisition device satisfies a first sub-conversion relationship with the terminal coordinate system of the labeling robot, and the base coordinate system satisfies a second sub-conversion relationship with the terminal coordinate system; the second conversion relationship includes a first sub-conversion relationship and a second sub-conversion relationship;

[0028] The parameter determination module is specifically used to determine the image position coordinates of the target workpiece in the second image coordinate system according to the second image; based on the first sub-transformation relationship, convert the image position coordinates into the first workpiece position coordinates in the terminal coordinate system; according to the second sub-transformation relationship, convert the first workpiece position coordinates into the second workpiece position coordinates in the base coordinate system, and use the second workpiece position coordinates as the second position parameter.

[0029] In one possible implementation manner, the second sub-conversion relationship is determined based on each joint angle of the labeling robot.

[0030] According to a third aspect of the present disclosure, there is provided an electronic device, including:

[0031] at least one processor; and

[0032] a memory communicatively connected to the at least one processor; wherein,

[0033] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method described in the present disclosure.

[0034] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable a computer to execute the method described in the present disclosure.

[0035] The label pasting method, device, equipment and storage medium disclosed in the present invention collect the first image of the label supply device based on the first image acquisition device, and collect the second image of the target workpiece based on the second image acquisition device; determine the first position parameter of the target label in the base coordinate system according to the first image, the first image coordinate system and the first conversion relationship, and determine the second position parameter of the target workpiece in the base coordinate system according to the second image, the second image coordinate system and the second conversion relationship; based on the first position parameter and the second position parameter, control the labeling robot to paste the target label on the target workpiece. That is, by pre-calibrating the labeling robot and the image acquisition device, the conversion relationship between each coordinate system and the base coordinate system of the labeling robot is obtained, and the coordinates of the target label and the target workpiece can be converted to the base coordinate system for alignment labeling, without the need to calibrate the tool coordinate system or the rotation center of the labeling robot, avoiding the complexity of the original coordinate conversion of various coordinate systems and the problem of large alignment calculation and the problem of poor labeling accuracy caused by wear of the labeling robot, simplifying the label pasting process and improving the label pasting accuracy.

[0036] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The above and other objects, features and advantages of the exemplary embodiments of the present disclosure will become readily understood by reading the detailed description below with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present disclosure are shown in an exemplary and non-limiting manner, in which:

[0038] In the drawings, the same or corresponding reference numerals represent the same or corresponding parts.

[0039] Figure 1 A schematic diagram of the implementation process of the label sticking method provided by the embodiment of the present disclosure is shown;

[0040] Figure 2 A schematic diagram of a label sticking device group is shown;

[0041] Figure 3 A schematic diagram of a parameter determination process provided by an embodiment of the present disclosure is shown;

[0042] Figure 4 Another parameter determination process diagram provided by an embodiment of the present disclosure is shown;

[0043] Figure 5 A schematic diagram of the structure of a label sticking device provided by an embodiment of the present disclosure is shown;

[0044] Figure 6 A schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0045] In order to make the purpose, features, and advantages of the present disclosure more obvious and easy to understand, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present disclosure.

[0046] Since the current labeling method has the problem of poor labeling accuracy caused by the position error of the workpiece to be labeled by the label stripping machine and the wear problem of the robot, in order to improve the labeling accuracy, the embodiments of the present disclosure provide a label pasting method, device, equipment and storage medium. The label pasting method provided by the present disclosure can be applied to electronic devices such as mobile phones, computers and tablet computers.

[0047] The technical solution of the embodiment of the present disclosure will be described below in conjunction with the drawings in the embodiment of the present disclosure.

[0048] Figure 1 The following is a schematic diagram of the implementation process of the label pasting method provided by the embodiment of the present disclosure. Figure 1 As shown, the method includes:

[0049] S101, capturing a first image of a label supply device based on a first image capturing device, and capturing a second image of a target workpiece based on a second image capturing device.

[0050] Among them, the first image acquisition device, the second image acquisition device and the labeling robot are pre-calibrated devices, the first image coordinate system of the first image acquisition device has a first conversion relationship with the base coordinate system of the labeling robot, and the second image coordinate system of the second image acquisition device has a second conversion relationship with the base coordinate system.

[0051] In this disclosure, Figure 2A schematic diagram of a label pasting device group is shown, such as Figure 2 As shown, a labeling robot 201, a first image acquisition device 202, a second image acquisition device 203, a label supply device 204, and a workpiece to be processed are configured on a production line for labeling workpieces. The target workpiece is a workpiece to be processed that needs to be labeled. The first image acquisition device and the second image acquisition device can be cameras or video cameras, etc. Figure 2 In the figure, point a represents the end part of the labeling robot 201. The end part of the labeling robot 201 is connected to the second image acquisition device 203 and the label suction device 205. The second image acquisition device 203 can be used to acquire the image of the target workpiece to determine the position of the target workpiece. The label suction device 205 can be used to absorb the target label from the label supply device 204 and paste the target label onto the target workpiece. The end component of the labeling robot follows the end coordinate system E, and the coordinate system of the robotic arm of the labeling robot can be controlled as the base coordinate system O.

[0052] In the present disclosure, the objects in the image captured by the first image acquisition device use the first image coordinate system, the objects in the image captured by the second image acquisition device use the second image coordinate system, and the terminal parts of the labeling robot follow the terminal coordinate system. In order to accurately control the labeling robot to paste the target label on the target workpiece, the first image acquisition device, the second image acquisition device and the labeling robot can be calibrated in advance so that each device follows the base coordinate system of the labeling robot. Specifically, in the present disclosure, the labeling robot can be kinematically calibrated, and the kinematic calibration can eliminate the influence of the kinematic parameter error caused by wear of the labeling robot during processing or long-term operation on the pasting of the label, thereby improving the accuracy of label pasting; in the present disclosure, the first image acquisition device and the second image acquisition device can also be internally calibrated to eliminate the internal parameter matrix error of the first image acquisition device and the second image acquisition device caused by processing; in the present disclosure, the image coordinate system and the base coordinate system can also be calibrated to determine the relationship between the image coordinate system and the base coordinate system. By pre-calibrating the first image acquisition device, the second image acquisition device and the labeling robot, the coordinates of the target label and the target workpiece can be converted to the base coordinate system of the labeling robot for alignment labeling control, thereby achieving precise labeling.

[0053] S102, determining the first position parameters of the target tag in the base coordinate system according to the first image, the first image coordinate system and the first transformation relationship, and determining the second position parameters of the target workpiece in the base coordinate system according to the second image, the second image coordinate system and the second transformation relationship.

[0054] S103: Based on the first position parameter and the second position parameter, control the labeling robot to attach the target label to the target workpiece.

[0055] According to the method, a first image of a label supply device is acquired based on a first image acquisition device, and a second image of a target workpiece is acquired based on a second image acquisition device; a first position parameter of a target label in a base coordinate system is determined based on the first image, the first image coordinate system and the first conversion relationship, and a second position parameter of a target workpiece in a base coordinate system is determined based on the second image, the second image coordinate system and the second conversion relationship; based on the first position parameter and the second position parameter, a labeling robot is controlled to paste the target label on the target workpiece. That is, by pre-calibrating the labeling robot and the image acquisition device, the conversion relationship between each coordinate system and the base coordinate system of the labeling robot is obtained, and the coordinates of the target label and the target workpiece can be converted to the base coordinate system for alignment labeling, without the need to calibrate the tool coordinate system or the rotation center of the labeling robot, thereby avoiding the complexity of coordinate conversion of various original coordinate systems, the large amount of alignment calculation, and the problem of poor labeling accuracy due to wear of the labeling robot, simplifying the label pasting process, and improving the label pasting accuracy.

[0056] In a possible implementation, kinematic calibration of the labeling robot may include: setting the coordinates of any point p on the end of the labeling robot in the base coordinate system to in, These are the DH parameters of the labeling robot. The DH parameters are used to describe the geometric relationship between the joints and connecting rods of the labeling robot. Characterizes the displacement distance of the adjacent link of the labeling robot along the x-axis of the previous link, that is, the horizontal distance along the x-axis, parameter Characterizes the rotation angle of the adjacent link of the labeling robot around the x-axis of the previous link, parameter Characterizes the displacement distance of the adjacent link of the labeling robot along the z-axis of the previous link, that is, the vertical distance along the z-axis, parameter Characterizes the rotation angle of the adjacent link of the labeling robot around the z-axis of the previous link. In order to perform kinematic calibration on the labeling robot, the labeling robot can be moved to any position, and the coordinate data of the end part of the labeling robot at the corresponding position in the end coordinate system and the data of each joint angle of the labeling robot can be recorded. According to this method, multiple sets of data of the labeling robot are measured, and then the Gauss-Newton method can be used to determine the series parameters in the nonlinear model:

[0057] Δf=JΔφ;

[0058] Where J is the Jacobian matrix of f, is the DH parameter error of the labeling robot. Δφ can be obtained by calculating the above formula using the least squares method:

[0059] Δφ=(J T J) -1 J T Δf

[0060] Then, by continuously iteratively updating Δφ, we get φ:

[0061] φ k+1 =φ k +Δφ

[0062] Where k is the number of iterations. When the error Δφ is small enough, the parameters obtained in φ can be used as the DH parameters of the calibrated labeling robot. The above calibration function relationship can be expressed as That is, the labeling robot is at each joint angle The DH parameters under this condition, the above calibration function relationship is also the second sub-transformation relationship satisfied between the base coordinate system and the terminal coordinate system.

[0063] In the present disclosure, the internal parameters of the first image acquisition device and the second image acquisition device may include parameters such as focal length, distortion parameters, principal point coordinates, and optical center offset. In a possible implementation, Figure 2 As shown, calibrating the internal parameters of the first image acquisition device and the second image acquisition device may include: for the first image acquisition device, a calibration plate may be placed above the first image acquisition device, the calibration plate may be changed to take photos in different postures to obtain multiple sets of calibration image data, and the internal parameters of the first image acquisition device are calculated based on the multiple sets of calibration image data; for the second image acquisition device, the calibration plate may be placed below the second image acquisition device, the calibration plate may be changed to take photos in different postures to obtain multiple sets of calibration image data, and the internal parameters of the second image acquisition device are calculated based on the multiple sets of calibration image data.

[0064] In a possible implementation, calibrating the image coordinate system and the base coordinate system to determine the relationship between the first image coordinate system and the base coordinate system may include:

[0065] For the first image acquisition device, the calibration plate can be adsorbed on the adsorption device of the labeling robot, and the first image acquisition device is used to collect multiple sets of image data of the calibration plate. The following formula can be used to calculate the first conversion relationship between the first image coordinate system of the first image acquisition device and the labeling robot based on the image data of the multiple sets of calibration plates:

[0066]

[0067] Among them, the first conversion relationship includes and and is a matrix representing the first transformation relationship between the first image coordinate system and the base coordinate system of the labeling robot, Indicates the coordinates of the calibration plate in the base coordinate system of the labeling robot. Indicates the coordinates of the calibration plate in the terminal coordinate system of the labeling robot. represents the coordinates of the calibration plate in the first image coordinate system, Represents the kinematic rotation matrix of the labeling robot itself, Indicates the coordinates of the end part of the labeling robot in the base coordinate system, is a fixed value, and the subscripts i and j represent any two calibration numbers.

[0068] Determining the relationship between the second image coordinate system and the base coordinate system may include:

[0069] For the second image acquisition device, the calibration plate can be placed on the label supply device, and the base coordinate system of the labeling robot and the position of the calibration plate are set to be fixed. Multiple sets of images of the calibration plate are collected, and the coordinates of any two calibration points on the calibration plate in the second image coordinate system are used to construct the following equation:

[0070]

[0071] in, and The matrix that represents the first sub-conversion relationship between the second image coordinate system and the terminal coordinate system of the labeling robot. The subscripts i and j represent the sequence numbers of any two calibrations. By moving the labeling robot and collecting multiple calibration data, the matrix included in the first sub-conversion relationship can be determined. and

[0072] The matrix included in determining the first conversion relationship between the first image coordinate system of the first image acquisition device and the base coordinate system of the labeling robot and And the first sub-conversion relationship between the second image coordinate system and the end coordinate system of the labeling robot and Afterwards, we can use the matrix and Determine the target conversion parameter, which is a parameter used to indicate the rotation angle and displacement required for the labeling robot to attach the target label to the target workpiece. In a possible implementation, the method for determining the target conversion parameter may include steps A1-A7:

[0073] Step A1, obtaining first coordinates of at least two position points in the target tag in a first image coordinate system.

[0074] In the present disclosure, the label supply device is used to place and store the target label. The first image acquisition device can be used to acquire an image of the label supply device on which the target label is placed, and the image coordinates of any two or more positions of the target label in the image are used as the first coordinates. Alternatively, if the first image includes an image of the target label, the image coordinates of any two or more positions of the target label in the first image can be used as the first coordinates.

[0075] For example, the first coordinate of the position point a1 in the target tag in the first image coordinate system can be obtained:

[0076]

[0077] Step A2: determining the second coordinates of each position point in the target tag in the base coordinate system based on the first conversion relationship and each first coordinate.

[0078] In the present disclosure, the matrix included in the first conversion relationship between the first image coordinate system and the base coordinate system can be used and The first coordinates of each position point in the target tag in the first image coordinate system are converted to the second coordinates in the base coordinate system. For example, the first coordinates of the position point a1 in the target tag in the first image coordinate system can be converted to the second coordinates in the base coordinate system by the following formula: Convert to the second coordinate in the base coordinate system

[0079]

[0080] Step A3, based on the first sub-conversion relationship and each second coordinate, determine the third coordinate of each position point in the target tag in the terminal coordinate system.

[0081] In the present disclosure, the first sub-conversion relationship satisfied between the base coordinate system and the terminal coordinate system can be The second coordinates of each position point in the target tag are converted to the third coordinates in the terminal coordinate system. For example, the second coordinates of the position point a1 in the target tag in the base coordinate system can be converted to Convert to the third coordinate in the terminal coordinate system

[0082]

[0083] Step A4, obtaining the first workpiece coordinates of the target position point on the target workpiece in the second image coordinate system.

[0084] The target position point on the target workpiece refers to the position point on the target workpiece for pasting the target label. In the present disclosure, a second image acquisition device can be used to acquire an image of the target workpiece, and the image coordinates of the target position point in the image are used as the first workpiece coordinates. Alternatively, if the second image includes an image of the target workpiece, the image coordinates of the target position point in the second image can be used as the first workpiece coordinates. For example, the first workpiece coordinates of the target position point b1 on the target workpiece in the second image coordinate system can be obtained.

[0085]

[0086] Step A5: determining the second workpiece coordinates of the target position point of the target workpiece in the terminal coordinate system based on the second sub-transformation relationship and the first workpiece coordinates.

[0087] In the present disclosure, the matrix included in the first sub-conversion relationship between the second image coordinate system and the terminal coordinate system can be used and Determine the second workpiece coordinate of the target position point of the target workpiece in the terminal coordinate system. For example, the second workpiece coordinate p of the target position point b1 on the target workpiece in the terminal coordinate system can be determined by the following formula: E :

[0088]

[0089] Step A6: Based on the second sub-transformation relationship and the second workpiece coordinate, determine the third workpiece coordinate of the target position point of the target workpiece in the base coordinate system.

[0090] In the present disclosure, the second sub-conversion relationship satisfied between the base coordinate system and the terminal coordinate system can be The second workpiece coordinate of the target position point of the target workpiece in the terminal coordinate system is converted into the third workpiece coordinate in the base coordinate system. For example, the second workpiece coordinate p of the target position point b1 on the target workpiece in the terminal coordinate system can be converted into the third workpiece coordinate in the base coordinate system by the following formula: E Transformed into the third workpiece coordinate system in the base coordinate system

[0091]

[0092] This is the joint angle vector of the labeling robot.

[0093] Step A7, determining the target conversion parameters according to the third coordinates of each position point in the target tag in the terminal coordinate system and the third workpiece coordinates of the target position point of the target workpiece in the base coordinate system.

[0094] In the present disclosure, after obtaining the third coordinates of each position point in the target tag in the terminal coordinate system and the third workpiece coordinates of the target position point of the target workpiece in the base coordinate system, the target conversion parameter can be determined using the following formula:

[0095]

[0096] in, and are target conversion parameters. and are all 2×n matrices, is a 2×2 matrix, is a 2×1 matrix, Characterizes the rotation angle parameter required for the labeling robot to stick the target label to the target workpiece. Characterizes the displacement parameters required by the labeling robot to paste the target label on the target workpiece. The displacement parameters include horizontal displacement parameters and vertical displacement parameters. That is, the target transformation parameters are used to indicate the rotation angle and displacement parameters required by the labeling robot to paste the target label on the target workpiece. and is the known value determined by the above conversion, The formula also includes three unknown parameters: rotation angle parameter, horizontal displacement parameter and vertical displacement parameter. Therefore, at least the coordinates of two position points in the target tag are required to determine the and Get the matrix including the target transformation relationship and Afterwards, you can follow and Control the labeling robot to adjust the rotation angle and displacement to stick the target label to the target workpiece.

[0097] In the present disclosure, there is a sequential relationship between step A1, step A3 and step A4.

[0098] There is a sequence relationship among step A4, step A5 and step A6, but there is no sequence relationship between step A4 and step A1, step A3 and step A3, there is no sequence relationship between step A5 and step A1, step A3 and step A3, and there is no sequence relationship between step A6 and step A1, step A3 and step A3.

[0099] The method provided by the embodiment of the present disclosure can simplify the process of labeling, eliminate the calibration of the tool coordinate system of the labeling robot, and avoid the calculation of the rotation offset between multiple coordinate systems, thereby improving the labeling accuracy and efficiency. In addition, since the calibration of the tool coordinate system of the labeling robot is eliminated, the method provided by the present disclosure can eliminate the labeling error caused by the non-fixed labeling position and the reduced kinematic accuracy of the labeling robot itself. In addition, the method provided by the present disclosure can meet the needs of production line changes and different labeling scenarios, with strong flexible production capacity and high algorithm universality.

[0100] In a possible implementation, the second image coordinate system of the second image acquisition device satisfies a first sub-transformation relationship with the terminal coordinate system of the labeling robot, and the base coordinate system and the terminal coordinate system satisfy a second sub-transformation relationship; the second transformation relationship includes a first sub-transformation relationship and a second sub-transformation relationship. Figure 3 A schematic diagram of a parameter determination process provided by an embodiment of the present disclosure is shown. Figure 3 As shown, determining the second position parameter of the target workpiece in the base coordinate system according to the second image, the second image coordinate system and the second conversion relationship includes:

[0101] S301 : determining the image position coordinates of the target workpiece in the second image coordinate system according to the second image.

[0102] In the present disclosure, since the internal parameters of the second image acquisition device are calibrated in advance, the internal parameters and distortion parameters of the second image acquisition device can be obtained, and the pixel coordinates of the target workpiece in the second image are converted into coordinates in the second image coordinate system according to the internal parameters and distortion parameters, that is, the pixel coordinates (x, y) of the target workpiece in the image are converted into normalized camera plane coordinates (u, v) using the internal parameters and distortion parameters.

[0103] S302: Based on the first sub-conversion relationship, convert the image position coordinates into first workpiece position coordinates in the terminal coordinate system.

[0104] The first sub-conversion relationship is a conversion matrix used to convert the coordinates in the second image coordinate system and the coordinates in the terminal coordinate system of the terminal part of the labeling robot. Through the first sub-conversion relationship, the first workpiece position coordinates of the target workpiece in the terminal coordinate system can be obtained.

[0105] S303: According to the second sub-conversion relationship, convert the first workpiece position coordinates into second workpiece position coordinates in the base coordinate system, and use the second workpiece position coordinates as second position parameters.

[0106] The second sub-transformation relationship is a transformation matrix for transforming the coordinates of the end coordinate system of the end part of the labeling robot with the coordinates of the base coordinate system of the labeling robot. Through the second sub-transformation relationship, the second workpiece position coordinates of the target workpiece in the base coordinate system can be further obtained. In the present disclosure, the second sub-transformation relationship is determined based on the joint angles of the labeling robot.

[0107] In a possible implementation, the first conversion relationship includes a third coordinate conversion matrix satisfied between the second image coordinate system of the second image acquisition device and the terminal coordinate system of the labeling robot. Figure 4 FIG. 4 shows another parameter determination process diagram provided by an embodiment of the present disclosure. Figure 4 As shown, determining the first position parameter of the target tag in the base coordinate system according to the first image, the first image coordinate system and the first conversion relationship includes:

[0108] S401: Determine the image position coordinates of the target tag in the first image coordinate system according to the first image.

[0109] In the present disclosure, since the first image acquisition device is calibrated with internal parameters in advance, the internal parameters and distortion parameters of the first image acquisition device can be obtained, and the pixel coordinates of the target label in the second image are converted into coordinates in the first image coordinate system according to the internal parameters and distortion parameters.

[0110] S402: Based on the third coordinate conversion matrix, convert the image position coordinates into label position coordinates in the base coordinate system, and use the label position coordinates as the first position parameter.

[0111] In the present disclosure, the third transformation matrix is ​​a transformation matrix between the first image coordinate system and the base coordinate system, and the label position coordinates in the base coordinate system can be obtained through the third coordinate transformation matrix.

[0112] In a possible implementation manner, controlling the labeling robot to attach the target label to the target workpiece based on the first position parameter and the second position parameter may include steps B1-B2:

[0113] Step B1, determining the offset distance and offset angle of the end of the labeling robot according to the first position parameter and the second position parameter.

[0114] Since the first position parameter and the second position parameter are both coordinates in the base coordinate system, the first horizontal displacement and the first vertical displacement between the end of the labeling robot and the label supply device, as well as the second horizontal displacement and the second vertical displacement between the end of the labeling robot and the target workpiece can be calculated based on the first position parameter and the second position parameter. The first horizontal displacement, the first vertical displacement, the second horizontal displacement and the second vertical displacement are all used as offset distances. The angle formed by the line connecting the end of the labeling robot and the origin of the base coordinate system and the line connecting the label supply device and the origin of the base coordinate system can also be calculated based on the first position parameter and the second position parameter as the offset angle.

[0115] Step B2: controlling the labeling robot to attach the target label to the target workpiece according to the offset distance and the offset angle.

[0116] According to the offset angle, the labeling robot is controlled to move the first horizontal displacement and the first vertical displacement distance to reach the label supply device, obtain the target label, and then the labeling robot is controlled to move the first vertical displacement and the first horizontal displacement in the opposite direction, and then the labeling robot is controlled to move the second horizontal displacement and the second vertical displacement distance to reach the target workpiece, and the labeling robot is controlled to stick the target label on the target workpiece.

[0117] Based on the same inventive concept, according to the label pasting method provided by the above embodiment of the present disclosure, correspondingly, another embodiment of the present disclosure further provides a label pasting device, whose structural schematic diagram is shown as follows: Figure 5 As shown, specifically including:

[0118] The image acquisition module 501 is used to acquire a first image of the label supply device based on a first image acquisition device, and to acquire a second image of the target workpiece based on a second image acquisition device; wherein the first image acquisition device, the second image acquisition device and the labeling robot are pre-calibrated devices, a first image coordinate system of the first image acquisition device and a base coordinate system of the labeling robot exist in a first conversion relationship, and a second image coordinate system of the second image acquisition device and the base coordinate system exist in a second conversion relationship;

[0119] a parameter determination module 502, configured to determine a first position parameter of the target tag in the base coordinate system according to the first image, the first image coordinate system and the first conversion relationship, and to determine a second position parameter of the target workpiece in the base coordinate system according to the second image, the second image coordinate system and the second conversion relationship;

[0120] The pasting module 503 is used to control the labeling robot to paste the target label onto the target workpiece based on the first position parameter and the second position parameter.

[0121] With the device, the first image of the label supply device is acquired based on the first image acquisition device, and the second image of the target workpiece is acquired based on the second image acquisition device; the first position parameter of the target label in the base coordinate system is determined based on the first image, the first image coordinate system and the first conversion relationship, and the second position parameter of the target workpiece in the base coordinate system is determined based on the second image, the second image coordinate system and the second conversion relationship; based on the first position parameter and the second position parameter, the labeling robot is controlled to paste the target label on the target workpiece. That is, by pre-calibrating the labeling robot and the image acquisition device, the conversion relationship between each coordinate system and the base coordinate system of the labeling robot is obtained, and the coordinates of the target label and the target workpiece can be converted to the base coordinate system for alignment labeling, without the need to calibrate the tool coordinate system or the rotation center of the labeling robot, avoiding the complexity of the original coordinate conversion of various coordinate systems and the problem of large alignment calculation and the problem of poor labeling accuracy caused by wear of the labeling robot, simplifying the label pasting process and improving the label pasting accuracy.

[0122] In one possible implementation manner, the second image coordinate system of the second image acquisition device satisfies a first sub-conversion relationship with the terminal coordinate system of the labeling robot, and the base coordinate system satisfies a second sub-conversion relationship with the terminal coordinate system; the second conversion relationship includes a first sub-conversion relationship and a second sub-conversion relationship;

[0123] The parameter determination module 502 is specifically used to determine the image position coordinates of the target workpiece in the second image coordinate system based on the second image; based on the first sub-transformation relationship, convert the image position coordinates into the first workpiece position coordinates in the terminal coordinate system; according to the second sub-transformation relationship, convert the first workpiece position coordinates into the second workpiece position coordinates in the base coordinate system, and use the second workpiece position coordinates as the second position parameter.

[0124] In one possible implementation manner, the second sub-conversion relationship is determined based on each joint angle of the labeling robot.

[0125] In one possible implementation, the first conversion relationship includes a third coordinate conversion matrix satisfied between the second image coordinate system of the second image acquisition device and the terminal coordinate system of the labeling robot;

[0126] The parameter determination module 502 is specifically used to determine the image position coordinates of the target label in the first image coordinate system according to the first image; based on the third coordinate transformation matrix, convert the image position coordinates into the label position coordinates in the base coordinate system, and use the label position coordinates as the first position parameter.

[0127] In one embodiment, the pasting module 503 is specifically used to determine the offset distance and offset angle of the end of the labeling robot according to the first position parameter and the second position parameter; and control the labeling robot to paste the target label onto the target workpiece according to the offset distance and the offset angle.

[0128] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device and a readable storage medium.

[0129] Figure 6 A schematic block diagram of an example electronic device 600 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0130] like Figure 6 As shown, the device 600 includes a computing unit 601, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the device 600 can also be stored. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0131] A number of components in the device 600 are connected to the I / O interface 605, including: an input unit 606, such as a keyboard, a mouse, etc.; an output unit 607, such as various types of displays, speakers, etc.; a storage unit 608, such as a disk, an optical disk, etc.; and a communication unit 609, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 609 allows the device 600 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0132] The computing unit 601 may be a variety of general and / or special processing development components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 601 performs the various methods and processes described above, such as the label pasting method. For example, in some embodiments, the label pasting method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 600 via ROM 602 and / or communication unit 609. When the computer program is loaded into RAM 603 and executed by the computing unit 601, one or more steps of the label pasting method described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to perform the label pasting method in any other appropriate manner (e.g., by means of firmware).

[0133] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0134] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0135] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0136] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0137] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0138] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0139] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0140] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of the present disclosure, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0141] The above is only a specific embodiment of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present disclosure, which should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. A label pasting method, characterized in that: The method comprises: A first image of the label supply device is acquired based on a first image acquisition device, and a second image of the target workpiece is acquired based on a second image acquisition device; wherein the first image acquisition device, the second image acquisition device and the labeling robot are pre-calibrated devices, a first image coordinate system of the first image acquisition device and a base coordinate system of the labeling robot have a first conversion relationship, and a second image coordinate system of the second image acquisition device and the base coordinate system have a second conversion relationship; Determine a first position parameter of the target tag in the base coordinate system according to the first image, the first image coordinate system and the first conversion relationship, and determine a second position parameter of the target workpiece in the base coordinate system according to the second image, the second image coordinate system and the second conversion relationship; Based on the first position parameter and the second position parameter, the labeling robot is controlled to attach the target label to the target workpiece.

2. The method according to claim 1, characterized in that The second image coordinate system of the second image acquisition device and the terminal coordinate system of the labeling robot satisfy a first sub-conversion relationship, and the base coordinate system and the terminal coordinate system satisfy a second sub-conversion relationship; the second conversion relationship includes a first sub-conversion relationship and a second sub-conversion relationship; Determining the second position parameter of the target workpiece in the base coordinate system according to the second image, the second image coordinate system and the second conversion relationship includes: Determine the image position coordinates of the target workpiece in the second image coordinate system according to the second image; Based on the first sub-conversion relationship, converting the image position coordinates into first workpiece position coordinates in the terminal coordinate system; According to the second sub-conversion relationship, the first workpiece position coordinates are converted into second workpiece position coordinates in the base coordinate system, and the second workpiece position coordinates are used as second position parameters.

3. The method according to claim 2, characterized in that The second sub-conversion relationship is determined based on each joint angle of the labeling robot.

4. The method according to claim 1, characterized in that The first conversion relationship includes a third coordinate conversion matrix satisfied between the second image coordinate system of the second image acquisition device and the terminal coordinate system of the labeling robot; The determining, according to the first image, the first image coordinate system and the first conversion relationship, a first position parameter of the target tag in the base coordinate system comprises: Determine the image position coordinates of the target tag in the first image coordinate system according to the first image; Based on the third coordinate conversion matrix, the image position coordinates are converted into label position coordinates in the base coordinate system, and the label position coordinates are used as the first position parameter.

5. The method according to claim 1, characterized in that The step of controlling the labeling robot to attach the target label to the target workpiece based on the first position parameter and the second position parameter includes: Determine the offset distance and the offset angle of the end of the labeling robot according to the first position parameter and the second position parameter; According to the offset distance and the offset angle, the labeling robot is controlled to attach the target label to the target workpiece.

6. A label sticking device, characterized in that: The device comprises: An image acquisition module, used for acquiring a first image of the label supply device based on a first image acquisition device, and acquiring a second image of the target workpiece based on a second image acquisition device; wherein the first image acquisition device, the second image acquisition device and the labeling robot are pre-calibrated devices, a first image coordinate system of the first image acquisition device and a base coordinate system of the labeling robot have a first conversion relationship, and a second image coordinate system of the second image acquisition device and the base coordinate system have a second conversion relationship; a parameter determination module, configured to determine a first position parameter of the target tag in the base coordinate system according to the first image, the first image coordinate system and the first conversion relationship, and to determine a second position parameter of the target workpiece in the base coordinate system according to the second image, the second image coordinate system and the second conversion relationship; A pasting module is used to control the labeling robot to paste the target label onto the target workpiece based on the first position parameter and the second position parameter.

7. The device according to claim 6, characterized in that The second image coordinate system of the second image acquisition device and the terminal coordinate system of the labeling robot satisfy a first sub-conversion relationship, and the base coordinate system and the terminal coordinate system satisfy a second sub-conversion relationship; the second conversion relationship includes a first sub-conversion relationship and a second sub-conversion relationship; The parameter determination module is specifically used to determine the image position coordinates of the target workpiece in the second image coordinate system according to the second image; based on the first sub-transformation relationship, convert the image position coordinates into the first workpiece position coordinates in the terminal coordinate system; according to the second sub-transformation relationship, convert the first workpiece position coordinates into the second workpiece position coordinates in the base coordinate system, and use the second workpiece position coordinates as the second position parameter.

8. The device according to claim 7, characterized in that The second sub-conversion relationship is determined based on each joint angle of the labeling robot.

9. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 5.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to make a computer execute the method according to any one of claims 1-5.

Images