A label sticking method, device, apparatus, and storage medium

Abstract

The present disclosure provides a label pasting method, device, equipment and storage medium. The method comprises: acquiring a first image of a label supply device based on a first image acquisition device, and acquiring a second image of a target workpiece based on a second image acquisition device; determining a first position parameter of a target label in a base coordinate system according to the first image, a first image coordinate system and a first conversion relationship, and determining a second position parameter of the target workpiece in the base coordinate system according to the second image, a second image coordinate system and a second conversion relationship; and controlling a label robot to paste the target label on the target workpiece based on the first position parameter and the second position parameter. The label pasting precision is improved by using the method.

Description

Technical Field

[0001] This disclosure relates to the field of computer processing, and more particularly to a label pasting method, apparatus, device, and storage medium. Background Technology

[0002] In the manufacturing process of electronic products, labeling is usually required. Labeling refers to affixing product labels containing information such as product model, production date, brand logo, and technical specifications to electronic products.

[0003] Common labeling methods include using vision sensors to guide robots for labeling. However, this method suffers from poor labeling accuracy due to variations in the position and angle of the label each time it is dispensed by the label peeler, errors in the position of the workpiece to be labeled, and wear and tear on the robot after long-term high-speed operation.

[0004] Therefore, improving labeling accuracy has become a pressing technical problem that needs to be solved. Summary of the Invention

[0005] This disclosure provides a label pasting method, apparatus, device, and storage medium to at least solve the above-mentioned technical problems existing in the prior art.

[0006] According to a first aspect of this disclosure, a label pasting method is provided, the method comprising:

[0007] A first image of the label supply device is acquired using a first image acquisition device, and a second image of the target workpiece is acquired using a second image acquisition device; wherein the first image acquisition device, the second image acquisition device, and the labeling robot are pre-calibrated devices, the first image coordinate system of the first image acquisition device and the base coordinate system of the labeling robot have a first transformation relationship, and the second image coordinate system of the second image acquisition device and the base coordinate system have a second transformation relationship;

[0008] Based on the first image, the first image coordinate system, and the first transformation relationship, the first position parameter of the target label in the base coordinate system is determined, and based on the second image, the second image coordinate system, and the second transformation relationship, the second position parameter of the target workpiece in the base coordinate system is determined.

[0009] Based on the first position parameter and the second position parameter, the labeling robot is controlled to affix the target label to the target workpiece.

[0010] In one embodiment, the second image coordinate system of the second image acquisition device and the end-effector coordinate system of the labeling robot satisfy a first sub-transformation relationship, and the base coordinate system and the end-effector coordinate system satisfy a second sub-transformation relationship; the second transformation relationship includes the first sub-transformation relationship and the second sub-transformation relationship;

[0011] The step of determining the second position parameters of the target workpiece in the base coordinate system based on the second image, the second image coordinate system, and the second transformation relationship includes:

[0012] Determine the image position coordinates of the target workpiece in the second image coordinate system based on the second image;

[0013] Based on the first sub-transformation relationship, the image position coordinates are converted into the first workpiece position coordinates in the end coordinate system;

[0014] According to the second sub-transformation relationship, the first workpiece position coordinates are converted into the second workpiece position coordinates in the base coordinate system, and the second workpiece position coordinates are used as the second position parameter.

[0015] In one possible implementation, the second sub-transformation relationship is determined based on the joint angles of the labeling robot.

[0016] In one possible implementation, the first transformation relationship includes a third coordinate transformation matrix satisfied between the second image coordinate system of the second image acquisition device and the end coordinate system of the labeling robot;

[0017] Determining the first position parameter of the target label in the base coordinate system based on the first image, the first image coordinate system, and the first transformation relationship includes:

[0018] Determine the image position coordinates of the target label in the first image coordinate system based on the first image;

[0019] Based on the third coordinate transformation matrix, the image position coordinates are converted into label position coordinates in the base coordinate system, and the label position coordinates are used as the first position parameter.

[0020] In one possible implementation, controlling the labeling robot to affix the target label to the target workpiece based on the first position parameter and the second position parameter includes:

[0021] The offset distance and offset angle of the labeling robot end are determined based on the first position parameter and the second position parameter;

[0022] Based on the offset distance and the offset angle, the labeling robot is controlled to affix the target label to the target workpiece.

[0023] According to a second aspect of this disclosure, a label affixing device is provided, the device comprising:

[0024] An image acquisition module is used to acquire a first image of the label supply device based on a first image acquisition device, and to acquire a second image of the target workpiece based on a second image acquisition device; wherein the first image acquisition device, the second image acquisition device, and the labeling robot are pre-calibrated devices, the first image coordinate system of the first image acquisition device and the base coordinate system of the labeling robot have a first transformation relationship, and the second image coordinate system of the second image acquisition device and the base coordinate system have a second transformation relationship;

[0025] The parameter determination module is used to determine the first position parameter of the target label in the base coordinate system based on the first image, the first image coordinate system and the first transformation relationship, and to determine the second position parameter of the target workpiece in the base coordinate system based on the second image, the second image coordinate system and the second transformation relationship.

[0026] The pasting module is used to control the labeling robot to paste the target label onto the target workpiece based on the first position parameter and the second position parameter.

[0027] In one embodiment, the second image coordinate system of the second image acquisition device and the end-effector coordinate system of the labeling robot satisfy a first sub-transformation relationship, and the base coordinate system and the end-effector coordinate system satisfy a second sub-transformation relationship; the second transformation relationship includes the first sub-transformation relationship and the second sub-transformation relationship;

[0028] The parameter determination module is specifically used to determine the image position coordinates of the target workpiece in the second image coordinate system based on the second image; convert the image position coordinates into the first workpiece position coordinates in the end coordinate system based on the first sub-transformation relationship; convert the first workpiece position coordinates into the second workpiece position coordinates in the base coordinate system based on the second sub-transformation relationship, and use the second workpiece position coordinates as the second position parameter.

[0029] In one possible implementation, the second sub-transformation relationship is determined based on the joint angles of the labeling robot.

[0030] According to a third aspect of this disclosure, an electronic device is provided, comprising:

[0031] At least one processor; and

[0032] A memory communicatively connected to the at least one processor; wherein,

[0033] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the methods described in this disclosure.

[0034] According to a fourth aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions for causing a computer to perform the methods described in this disclosure.

[0035] The label pasting method, apparatus, device, and storage medium disclosed herein acquire a first image of a label supply device using a first image acquisition device, and a second image of a target workpiece using a second image acquisition device. Based on the first image, a first image coordinate system, and a first transformation relationship, a first position parameter of the target label in a base coordinate system is determined; and based on the second image, a second image coordinate system, and a second transformation relationship, a second position parameter of the target workpiece in the base coordinate system is determined. Based on the first and second position parameters, a labeling robot is controlled to paste the target label onto the target workpiece. In other words, by pre-calibrating the labeling robot and the image acquisition device, the transformation relationship between each coordinate system and the base coordinate system of the labeling robot is obtained. This allows the coordinates of both the target label and the target workpiece to be transformed to the base coordinate system for alignment and labeling, eliminating the need for calibration of the labeling robot's tool coordinate system or rotation center. This avoids the complexity of coordinate transformations and the large amount of alignment calculations inherent in traditional coordinate systems, as well as the poor labeling accuracy caused by wear on the labeling robot. This simplifies the label pasting process and improves label pasting accuracy.

[0036] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0037] The above and other objects, features, and advantages of this disclosure will become readily apparent from the following detailed description of exemplary embodiments, taken in conjunction with the accompanying drawings. Several embodiments of this disclosure are illustrated in the drawings by way of example and not limitation, in which:

[0038] In the accompanying drawings, the same or corresponding reference numerals indicate the same or corresponding parts.

[0039] Figure 1 A schematic diagram illustrating the implementation flow of the label pasting method provided in this embodiment of the present disclosure is shown;

[0040] Figure 2 A schematic diagram of a label-adhesive device assembly is shown;

[0041] Figure 3 This illustration shows a parameter determination process according to an embodiment of the present disclosure;

[0042] Figure 4 This illustration shows another parameter determination process provided by an embodiment of the present disclosure;

[0043] Figure 5 A schematic diagram of the label pasting device provided in an embodiment of this disclosure is shown;

[0044] Figure 6 A schematic diagram of the composition structure of an electronic device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0045] To make the objectives, features, and advantages of this disclosure more apparent and understandable, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0046] Current labeling methods suffer from poor labeling accuracy due to errors in the position of the workpiece to be labeled by the label peeler and wear and tear on the robot. Therefore, to improve labeling accuracy, this disclosure provides a label pasting method, apparatus, device, and storage medium. The label pasting method provided in this disclosure can be applied to electronic devices such as mobile phones, computers, and tablet computers.

[0047] The technical solutions of the embodiments of this disclosure will now be described with reference to the accompanying drawings.

[0048] Figure 1 A schematic diagram illustrating the implementation flow of the label pasting method provided in this embodiment of the present disclosure is shown, as follows: Figure 1 As shown, the method includes:

[0049] S101, a first image of the tag supply device is acquired based on the first image acquisition device, and a second image of the target workpiece is acquired based on the second image acquisition device.

[0050] The first image acquisition device, the second image acquisition device, and the labeling robot are pre-calibrated devices. The first image coordinate system of the first image acquisition device has a first transformation relationship with the base coordinate system of the labeling robot, and the second image coordinate system of the second image acquisition device has a second transformation relationship with the base coordinate system.

[0051] In this disclosure, Figure 2A schematic diagram of a label-adhesive device assembly is shown, such as Figure 2 As shown, the production line for affixing labels to workpieces is equipped with a labeling robot 201, a first image acquisition device 202, a second image acquisition device 203, a label supply device 204, and workpieces to be processed. The target workpiece is the workpiece among the workpieces to be processed that requires label affixing. The first and second image acquisition devices can be cameras or video cameras, etc. Figure 2 In the diagram, point a represents the end part of the labeling robot 201. The end part of the labeling robot 201 is connected to the second image acquisition device 203 and the label suction device 205. The second image acquisition device 203 can be used to acquire images of the target workpiece to determine the position of the target workpiece. The label suction device 205 can be used to absorb the target label from the label supply device 204 and stick the target label onto the target workpiece. The end part of the labeling robot follows the end coordinate system E, and the coordinate system of the robotic arm of the labeling robot can be controlled as the base coordinate system O.

[0052] In this disclosure, objects in images captured by the first image acquisition device use a first image coordinate system, objects in images captured by the second image acquisition device use a second image coordinate system, and the end effector of the labeling robot follows the end effector coordinate system. To accurately control the labeling robot to affix the target label to the target workpiece, the first image acquisition device, the second image acquisition device, and the labeling robot can be pre-calibrated so that each device follows the labeling robot's base coordinate system. Specifically, this disclosure allows for kinematic calibration of the labeling robot. Kinematic calibration eliminates the influence of kinematic parameter errors caused by wear during processing or long-term operation on label affixing, thereby improving label affixing accuracy. This disclosure also allows for intrinsic parameter calibration of the first and second image acquisition devices to eliminate intrinsic parameter matrix errors caused by processing. Furthermore, this disclosure allows for calibration of the image coordinate system and the base coordinate system to determine the relationship between them. By pre-calibrating the first image acquisition device, the second image acquisition device, and the labeling robot, the coordinates of the target label and the target workpiece can be transformed into the base coordinate system of the labeling robot for alignment and labeling control, thus achieving precise labeling.

[0053] S102, based on the first image, the first image coordinate system, and the first transformation relationship, determine the first position parameter of the target label in the base coordinate system, and based on the second image, the second image coordinate system, and the second transformation relationship, determine the second position parameter of the target workpiece in the base coordinate system.

[0054] S103, based on the first position parameter and the second position parameter, control the labeling robot to paste the target label onto the target workpiece.

[0055] This method involves acquiring a first image of the label supply device using a first image acquisition device, and a second image of the target workpiece using a second image acquisition device. Based on the first image, the first image coordinate system, and a first transformation relationship, a first position parameter of the target label in the base coordinate system is determined. Similarly, based on the second image, the second image coordinate system, and a second transformation relationship, a second position parameter of the target workpiece in the base coordinate system is determined. Based on the first and second position parameters, the labeling robot is controlled to affix the target label to the target workpiece. In other words, by pre-calibrating the labeling robot and the image acquisition device, the transformation relationship between each coordinate system and the base coordinate system of the labeling robot is obtained. This allows the coordinates of both the target label and the target workpiece to be transformed to the base coordinate system for alignment and labeling, eliminating the need for calibration of the labeling robot's tool coordinate system or rotation center. This avoids the complexity of coordinate transformations and the large amount of alignment calculations inherent in traditional methods, as well as the poor labeling accuracy caused by wear on the labeling robot. This simplifies the labeling process and improves labeling accuracy.

[0056] In one possible implementation, kinematic calibration of the labeling robot may include: setting the coordinates of any point p on the end effector of the labeling robot in the base coordinate system as follows: in, These are all DH parameters for the labeling robot. DH parameters are used to describe the geometric relationships between the joints and links of the labeling robot. The parameter represents the horizontal distance along the x-axis of the adjacent link of the labeling robot, i.e., the distance along the x-axis. The parameter represents the rotation angle of an adjacent link of the labeling robot about the x-axis of the previous link. The parameter represents the displacement distance of an adjacent link of a labeling robot along the z-axis of the previous link, i.e., the vertical distance along the z-axis. This represents the rotation angle of adjacent links of the labeling robot about the z-axis of the preceding link. To perform kinematic calibration of the labeling robot, it can be moved to any position, and the coordinate data of the end effector in the end effector coordinate system and the joint angle data of the robot at that position can be recorded. Multiple sets of data for the labeling robot are obtained using this method, and then the Gauss-Newton method can be used to determine a series of parameters in the nonlinear model:

[0057] Δf = JΔφ;

[0058] Where J is the Jacobian matrix of f. The DH parameter error of the labeling robot can be calculated using the least squares method to obtain Δφ from the above formula:

[0059] Δφ=(J T J) -1 J T Δf

[0060] Then, by continuously iterating and updating Δφ, we obtain φ:

[0061] φ k+1 =φ k +Δφ

[0062] Where k is the number of iterations, when the error Δφ is sufficiently small, the parameters obtained in φ can be used as the DH parameters of the calibrated labeling robot. The above calibration function relationship can be expressed as follows: The labeling robot at each joint angle The DH parameters below, the above calibration function relationship is also the second sub-transformation relationship satisfied between the base coordinate system and the end coordinate system.

[0063] In this disclosure, the intrinsic parameters of the first image acquisition device and the second image acquisition device may include parameters such as focal length, distortion parameters, principal point coordinates, and optical center offset. In one possible implementation, such as... Figure 2 As shown, the intrinsic parameter calibration of the first image acquisition device and the second image acquisition device may include: for the first image acquisition device, a calibration plate can be placed above the first image acquisition device, and the calibration plate can be changed to take pictures in different postures to obtain multiple sets of calibration image data, and the intrinsic parameters of the first image acquisition device can be calculated based on the multiple sets of calibration image data; for the second image acquisition device, a calibration plate can be placed below the second image acquisition device, and the calibration plate can be changed to take pictures in different postures to obtain multiple sets of calibration image data, and the intrinsic parameters of the second image acquisition device can be calculated based on the multiple sets of calibration image data.

[0064] In one possible implementation, calibrating the image coordinate system and the base coordinate system to determine the relationship between the first image coordinate system and the base coordinate system may include:

[0065] For the first image acquisition device, the calibration plate can be adsorbed onto the adsorption device of the labeling robot. Multiple sets of image data from the calibration plate can be acquired using the first image acquisition device. The following formula can be used to calculate the first transformation relationship between the first image coordinate system of the first image acquisition device and the labeling robot based on the multiple sets of image data from the calibration plate:

[0066]

[0067] The first transformation relationship includes and and A matrix representing the first transformation relationship between the first image coordinate system and the base coordinate system of the labeling robot. This indicates the coordinates of the calibration plate in the base coordinate system of the labeling robot. This indicates the coordinates of the calibration plate in the end-effector coordinate system of the labeling robot. This indicates the coordinates of the calibration plate in the first image coordinate system. This represents the kinematic rotation matrix of the labeling robot itself. This represents the coordinates of the end effector of the labeling robot in the base coordinate system. The values ​​are fixed, and the subscripts i and j represent any two calibration sequences.

[0068] Determining the relationship between the second image coordinate system and the base coordinate system can include:

[0069] For the second image acquisition device, the calibration plate can be placed on the label supply device. The base coordinate system of the labeling robot and the position of the calibration plate are set to remain fixed. Multiple sets of images of the calibration plate are acquired. The following equation is constructed using the coordinates of any two calibration points on the calibration plate in the second image coordinate system:

[0070]

[0071] in, and To characterize the first sub-transformation relationship between the second image coordinate system and the end-effector coordinate system of the labeling robot, the subscripts i and j represent any two calibration sequences. By moving the labeling robot and collecting multiple calibration data, the matrix included in the first sub-transformation relationship can be determined. and

[0072] The matrix included in determining the first transformation relationship between the first image coordinate system of the first image acquisition device and the base coordinate system of the labeling robot. and And the first sub-transformation relationship between the second image coordinate system and the end effector coordinate system of the labeling robot. and Then, based on the matrix and Determine the target transformation parameters, which are parameters used to instruct the labeling robot on the rotation angle and displacement required to affix the target label to the target workpiece. In one possible implementation, the method for determining the target transformation parameters may include steps A1-A7:

[0073] Step A1: Obtain the first coordinates of at least two location points in the target label in the first image coordinate system.

[0074] In this disclosure, the tag supply device is used to place and store target tags. It can employ a first image acquisition device to acquire an image of the tag supply device with the target tags placed thereon, and use the image coordinates of any two or more locations of the target tags in the image as the first coordinates. Alternatively, if the first image includes an image of the target tags, the image coordinates of any two or more locations of the target tags in the first image can be used as the first coordinates.

[0075] For example, the first coordinates of position point a1 in the target label in the first image coordinate system can be obtained.

[0076]

[0077] Step A2: Based on the first transformation relationship and each first coordinate, determine the second coordinates of each location point in the target label in the base coordinate system.

[0078] In this disclosure, the matrix included in the first transformation relationship between the first image coordinate system and the base coordinate system can be utilized. and Convert the first coordinates of each location point in the target label from the first image coordinate system to the second coordinates in the base coordinate system. For example, the following formula can be used to convert the first coordinates of location point a1 in the target label from the first image coordinate system to the second coordinates. Transform into the second coordinate system in the base coordinate system

[0079]

[0080] Step A3: Based on the first sub-transformation relationship and each second coordinate, determine the third coordinates of each location point in the target label in the end coordinate system.

[0081] In this disclosure, the first sub-transformation relation satisfied between the base coordinate system and the terminal coordinate system can be used. Convert the second coordinates of each location point in the target label to third coordinates in the end coordinate system. For example, the second coordinates of location point a1 in the target label in the base coordinate system can be converted using the following formula. Transform into third coordinates in the end coordinate system

[0082]

[0083] Step A4: Obtain the first workpiece coordinates of the target position point on the target workpiece in the second image coordinate system.

[0084] The target position point on the target workpiece refers to the position point on the target workpiece used for affixing the target label. In this disclosure, a second image acquisition device can be used to acquire an image of the target workpiece, and the image coordinates of the target position point in the image can be used as the first workpiece coordinates. Alternatively, if the second image includes an image of the target workpiece, the image coordinates of all target position points in the second image can be used as the first workpiece coordinates. For example, the first workpiece coordinates of the target position point b1 on the target workpiece in the second image coordinate system can be obtained.

[0085]

[0086] Step A5: Based on the second sub-transformation relationship and the first workpiece coordinates, determine the second workpiece coordinates of the target position point of the target workpiece in the end coordinate system.

[0087] In this disclosure, the matrix included in the first sub-transformation relationship between the second image coordinate system and the end coordinate system can be utilized. and Determine the second workpiece coordinates of the target position point b1 on the target workpiece in the end coordinate system. For example, the second workpiece coordinates p of the target position point b1 on the target workpiece in the end coordinate system can be determined using the following formula. E :

[0088]

[0089] Step A6: Based on the second sub-transformation relationship and the second workpiece coordinates, determine the third workpiece coordinates of the target position point in the base coordinate system.

[0090] In this disclosure, a second sub-transformation relation satisfied between the base coordinate system and the terminal coordinate system can be used. The second workpiece coordinates of the target position point b1 on the target workpiece in the end coordinate system are converted to the third workpiece coordinates in the base coordinate system. For example, the following formula can be used to convert the second workpiece coordinates p of the target position point b1 on the target workpiece in the end coordinate system. E Transform into the coordinates of the third workpiece in the base coordinate system

[0091]

[0092] This is the joint angle vector of the labeling robot at this time.

[0093] Step A7: Determine the target transformation parameters based on the third coordinates of each position point in the target label in the end coordinate system and the third workpiece coordinates of the target position point in the base coordinate system.

[0094] In this disclosure, after obtaining the third coordinates of each position point in the target label in the end coordinate system and the third workpiece coordinates of the target position point in the base coordinate system, the target transformation parameters can be determined using the following formula:

[0095]

[0096] in, and All are target transformation parameters. and Both are 2×n matrices. It is a 2×2 matrix. It is a 2×1 matrix. This describes the rotation angle parameters required for the labeling robot to attach the target label to the target workpiece. The displacement parameters characterize the distance the labeling robot needs to attach the target label to the target workpiece. These displacement parameters include horizontal and vertical displacement parameters; in other words, the target transformation parameters are the parameters that indicate the rotation angle and displacement required for the labeling robot to attach the target label to the target workpiece. Because... and The known values ​​determined through the above transformations, The formula also includes three unknown parameters: rotation angle, horizontal displacement, and vertical displacement. Therefore, the coordinates of at least two points in the target label are needed to determine the position. and The matrix containing the target transformation relationship is obtained. and After that, you can follow and The labeling robot is controlled to adjust its rotation angle and displacement to attach the target label to the target workpiece.

[0097] In this disclosure, steps A1, A3, and A4 are sequentially related.

[0098] Steps A4, A5, and A6 are sequential, but steps A4, A1, A3, and A3 are not sequential; steps A5, A1, A3, and A3 are not sequential; and steps A6, A1, A3, and A3 are not sequential.

[0099] The method provided in this disclosure simplifies the labeling process, eliminates the need for calibration of the labeling robot's tool coordinate system, and avoids intermediate calculations of rotational offsets between multiple coordinate systems, thus improving labeling accuracy and efficiency. Furthermore, by eliminating the need for tool coordinate system calibration, the method eliminates labeling errors caused by inconsistent label placement positions or reduced kinematic accuracy of the labeling robot itself. Moreover, the method provided in this disclosure can meet the needs of production line changes and different labeling scenarios, demonstrating strong flexible production capabilities and high algorithm versatility.

[0100] In one possible implementation, the second image coordinate system of the second image acquisition device and the end coordinate system of the labeling robot satisfy a first sub-transformation relationship, and the base coordinate system and the end coordinate system satisfy a second sub-transformation relationship; the second transformation relationship includes the first sub-transformation relationship and the second sub-transformation relationship. Figure 3 A schematic diagram of a parameter determination process provided by an embodiment of this disclosure is shown, such as... Figure 3 As shown, determining the second position parameters of the target workpiece in the base coordinate system based on the second image, the second image coordinate system, and the second transformation relationship includes:

[0101] S301, determine the image position coordinates of the target workpiece in the second image coordinate system based on the second image.

[0102] In this disclosure, since the intrinsic parameters of the second image acquisition device are calibrated in advance, the intrinsic parameters and distortion parameters of the second image acquisition device can be obtained. Based on the intrinsic parameters and distortion parameters, the pixel coordinates of the target workpiece in the second image are converted into coordinates in the second image coordinate system. That is, the pixel coordinates (x, y) of the target workpiece in the image are converted into normalized camera plane coordinates (u, v) using the intrinsic parameters and distortion parameters.

[0103] S302, based on the first sub-transformation relationship, the image position coordinates are converted into the first workpiece position coordinates in the end coordinate system.

[0104] The first sub-transformation relation is a transformation matrix used to transform the coordinates in the second image coordinate system and the coordinates in the end-effector coordinate system of the labeling robot. Through the first sub-transformation relation, the first workpiece position coordinates in the end-effector coordinate system can be obtained.

[0105] S303, according to the second sub-transformation relationship, the first workpiece position coordinates are converted into the second workpiece position coordinates in the base coordinate system, and the second workpiece position coordinates are used as the second position parameter.

[0106] The second sub-transformation relationship is a transformation matrix used to convert the coordinates of the end effector of the labeling robot in its end-effector coordinate system to the coordinates in its base coordinate system. Through this second sub-transformation relationship, the second workpiece position coordinates in the base coordinate system can be further obtained. In this disclosure, the second sub-transformation relationship is determined based on the joint angles of the labeling robot.

[0107] In one possible implementation, the first transformation relationship includes a third coordinate transformation matrix satisfied between the second image coordinate system of the second image acquisition device and the end coordinate system of the labeling robot. Figure 4 This illustration shows another parameter determination process provided by an embodiment of the present disclosure, such as... Figure 4 As shown, determining the first position parameter of the target label in the base coordinate system based on the first image, the first image coordinate system, and the first transformation relationship includes:

[0108] S401, determine the image position coordinates of the target label in the first image coordinate system based on the first image.

[0109] In this disclosure, since the intrinsic parameters of the first image acquisition device have been calibrated in advance, the intrinsic parameters and distortion parameters of the first image acquisition device can be obtained, and the pixel coordinates of the target label in the second image can be converted into coordinates in the first image coordinate system based on the intrinsic parameters and distortion parameters.

[0110] S402, based on the third coordinate transformation matrix, the image position coordinates are converted into label position coordinates in the base coordinate system, and the label position coordinates are used as the first position parameter.

[0111] In this disclosure, the third transformation matrix is ​​the transformation matrix between the first image coordinate system and the base coordinate system. The label position coordinates in the base coordinate system can be obtained through the third coordinate transformation matrix.

[0112] In one possible implementation, controlling the labeling robot to affix the target label to the target workpiece based on the first position parameter and the second position parameter may include steps B1-B2:

[0113] Step B1: Determine the offset distance and offset angle of the labeling robot end based on the first position parameter and the second position parameter.

[0114] Since both the first and second position parameters are coordinates in a base coordinate system, the first horizontal and first vertical displacements between the labeling robot's end effector and the label supply device, as well as the second horizontal and second vertical displacements between the labeling robot's end effector and the target workpiece, can be calculated based on these parameters. The first horizontal, first vertical, second horizontal, and second vertical displacements are all considered as offset distances. Alternatively, the angle formed by the line connecting the labeling robot's end effector and the origin of the base coordinate system, and the line connecting the label supply device and the origin of the base coordinate system, can be calculated using the first and second position parameters and used as the offset angle.

[0115] Step B2: Based on the offset distance and the offset angle, control the labeling robot to affix the target label to the target workpiece.

[0116] Based on the offset angle, the labeling robot is controlled to move a first horizontal displacement and a first vertical displacement distance to reach the label supply device, obtain the target label, and then the labeling robot is controlled to move in the opposite direction by the first vertical displacement and the first horizontal displacement. Then, the labeling robot is controlled to move a second horizontal displacement and a second vertical displacement distance to reach the target workpiece, and the labeling robot is controlled to paste the target label on the target workpiece.

[0117] Based on the same inventive concept, and according to the label pasting method provided in the above embodiments of this disclosure, another embodiment of this disclosure also provides a label pasting device, the structural schematic diagram of which is shown below. Figure 5 As shown, it specifically includes:

[0118] Image acquisition module 501 is used to acquire a first image of the label supply device based on a first image acquisition device, and to acquire a second image of the target workpiece based on a second image acquisition device; wherein the first image acquisition device, the second image acquisition device, and the labeling robot are pre-calibrated devices, the first image coordinate system of the first image acquisition device and the base coordinate system of the labeling robot have a first transformation relationship, and the second image coordinate system of the second image acquisition device and the base coordinate system have a second transformation relationship;

[0119] The parameter determination module 502 is used to determine the first position parameter of the target label in the base coordinate system according to the first image, the first image coordinate system and the first transformation relationship, and to determine the second position parameter of the target workpiece in the base coordinate system according to the second image, the second image coordinate system and the second transformation relationship.

[0120] The pasting module 503 is used to control the labeling robot to paste the target label onto the target workpiece based on the first position parameter and the second position parameter.

[0121] This device acquires a first image of the label supply device using a first image acquisition device, and a second image of the target workpiece using a second image acquisition device. Based on the first image, the first image coordinate system, and a first transformation relationship, it determines the first position parameters of the target label in the base coordinate system. Based on the second image, the second image coordinate system, and a second transformation relationship, it determines the second position parameters of the target workpiece in the base coordinate system. Based on the first and second position parameters, it controls the labeling robot to affix the target label to the target workpiece. In other words, by pre-calibrating the labeling robot and the image acquisition device, the transformation relationships between various coordinate systems and the labeling robot's base coordinate system are obtained. This allows the coordinates of both the target label and the target workpiece to be transformed to the base coordinate system for alignment and labeling, eliminating the need for calibration of the labeling robot's tool coordinate system or rotation center. This avoids the complexity of coordinate transformations and the large amount of alignment calculations inherent in traditional methods, as well as the poor labeling accuracy caused by wear on the labeling robot. This simplifies the labeling process and improves labeling accuracy.

[0122] In one embodiment, the second image coordinate system of the second image acquisition device and the end-effector coordinate system of the labeling robot satisfy a first sub-transformation relationship, and the base coordinate system and the end-effector coordinate system satisfy a second sub-transformation relationship; the second transformation relationship includes the first sub-transformation relationship and the second sub-transformation relationship;

[0123] The parameter determination module 502 is specifically used to determine the image position coordinates of the target workpiece in the second image coordinate system based on the second image; convert the image position coordinates into the first workpiece position coordinates in the end coordinate system based on the first sub-transformation relationship; convert the first workpiece position coordinates into the second workpiece position coordinates in the base coordinate system based on the second sub-transformation relationship, and use the second workpiece position coordinates as the second position parameter.

[0124] In one possible implementation, the second sub-transformation relationship is determined based on the joint angles of the labeling robot.

[0125] In one possible implementation, the first transformation relationship includes a third coordinate transformation matrix satisfied between the second image coordinate system of the second image acquisition device and the end coordinate system of the labeling robot;

[0126] The parameter determination module 502 is specifically used to determine the image position coordinates of the target label in the first image coordinate system based on the first image; and to convert the image position coordinates into label position coordinates in the base coordinate system based on the third coordinate transformation matrix, and to use the label position coordinates as the first position parameter.

[0127] In one embodiment, the pasting module 503 is specifically used to determine the offset distance and offset angle of the end of the labeling robot according to the first position parameter and the second position parameter; and to control the labeling robot to paste the target label onto the target workpiece according to the offset distance and the offset angle.

[0128] According to embodiments of this disclosure, this disclosure also provides an electronic device and a readable storage medium.

[0129] Figure 6 A schematic block diagram of an example electronic device 600 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0130] like Figure 6 As shown, device 600 includes a computing unit 601, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 602 or a computer program loaded from storage unit 608 into random access memory (RAM) 603. RAM 603 may also store various programs and data required for the operation of device 600. The computing unit 601, ROM 602, and RAM 603 are interconnected via bus 604. Input / output (I / O) interface 605 is also connected to bus 604.

[0131] Multiple components in device 600 are connected to I / O interface 605, including: input unit 606, such as keyboard, mouse, etc.; output unit 607, such as various types of monitors, speakers, etc.; storage unit 608, such as disk, optical disk, etc.; and communication unit 609, such as network card, modem, wireless transceiver, etc. Communication unit 609 allows device 600 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0132] The computing unit 601 can be a variety of general-purpose and / or special-purpose processing development components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 601 performs the various methods and processes described above, such as the label pasting method. For example, in some embodiments, the label pasting method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on device 600 via ROM 602 and / or communication unit 609. When the computer program is loaded into RAM 603 and executed by the computing unit 601, one or more steps of the label pasting method described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to perform the label pasting method by any other suitable means (e.g., by means of firmware).

[0133] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0134] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0135] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0136] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0137] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0138] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0139] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0140] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this disclosure, "a plurality of" means two or more, unless otherwise explicitly specified.

[0141] The above description is merely a specific embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this disclosure should be included within the scope of protection of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the scope of the claims.

Claims

1. A label pasting method, characterized in that, The method includes: A first image of the label supply device is acquired using a first image acquisition device, and a second image of the target workpiece is acquired using a second image acquisition device; wherein the first image acquisition device, the second image acquisition device, and the labeling robot are pre-calibrated devices, the first image coordinate system of the first image acquisition device and the base coordinate system of the labeling robot have a first transformation relationship, and the second image coordinate system of the second image acquisition device and the base coordinate system have a second transformation relationship; Based on the first image, the first image coordinate system, and the first transformation relationship, determine the first position parameter of the target label in the base coordinate system; and based on the second image, the second image coordinate system, and the second transformation relationship, determine the second position parameter of the target workpiece in the base coordinate system. Based on the first position parameter and the second position parameter located in the base coordinate system, the offset distance and offset angle of the labeling robot end are determined; Based on the offset distance and offset angle of the labeling robot's end effector, the labeling robot is controlled to affix the target label to the target workpiece.

2. The method according to claim 1, characterized in that, The second image coordinate system of the second image acquisition device and the end coordinate system of the labeling robot satisfy a first sub-transformation relationship, and the base coordinate system and the end coordinate system satisfy a second sub-transformation relationship; the second transformation relationship includes the first sub-transformation relationship and the second sub-transformation relationship; The step of determining the second position parameters of the target workpiece in the base coordinate system based on the second image, the second image coordinate system, and the second transformation relationship includes: Determine the image position coordinates of the target workpiece in the second image coordinate system based on the second image; Based on the first sub-transformation relationship, the image position coordinates are converted into the first workpiece position coordinates in the end coordinate system; According to the second sub-transformation relationship, the first workpiece position coordinates are converted into the second workpiece position coordinates in the base coordinate system, and the second workpiece position coordinates are used as the second position parameter.

3. The method according to claim 2, characterized in that, The second sub-transformation relationship is determined based on the joint angles of the labeling robot.

4. The method according to claim 1, characterized in that, The first transformation relationship includes the third coordinate transformation matrix satisfied between the second image coordinate system of the second image acquisition device and the end coordinate system of the labeling robot; Determining the first position parameter of the target label in the base coordinate system based on the first image, the first image coordinate system, and the first transformation relationship includes: Determine the image position coordinates of the target label in the first image coordinate system based on the first image; Based on the third coordinate transformation matrix, the image position coordinates are converted into label position coordinates in the base coordinate system, and the label position coordinates are used as the first position parameter.

5. A label pasting device, characterized in that, The device includes: An image acquisition module is used to acquire a first image of the label supply device based on a first image acquisition device, and to acquire a second image of the target workpiece based on a second image acquisition device; wherein the first image acquisition device, the second image acquisition device, and the labeling robot are pre-calibrated devices, the first image coordinate system of the first image acquisition device and the base coordinate system of the labeling robot have a first transformation relationship, and the second image coordinate system of the second image acquisition device and the base coordinate system have a second transformation relationship; The parameter determination module is used to determine a first position parameter of the target label in the base coordinate system based on the first image, the first image coordinate system and the first transformation relationship, and to determine a second position parameter of the target workpiece in the base coordinate system based on the second image, the second image coordinate system and the second transformation relationship. The pasting module is used to determine the offset distance and offset angle of the labeling robot end based on the first position parameter and the second position parameter located in the base coordinate system; The pasting module is also used to control the labeling robot to paste the target label onto the target workpiece based on the offset distance and offset angle of the end of the labeling robot.

6. The apparatus according to claim 5, characterized in that, The second image coordinate system of the second image acquisition device and the end coordinate system of the labeling robot satisfy a first sub-transformation relationship, and the base coordinate system and the end coordinate system satisfy a second sub-transformation relationship; the second transformation relationship includes the first sub-transformation relationship and the second sub-transformation relationship; The parameter determination module is specifically used to determine the image position coordinates of the target workpiece in the second image coordinate system based on the second image; convert the image position coordinates into the first workpiece position coordinates in the end coordinate system based on the first sub-transformation relationship; convert the first workpiece position coordinates into the second workpiece position coordinates in the base coordinate system based on the second sub-transformation relationship, and use the second workpiece position coordinates as the second position parameter.

7. The apparatus according to claim 6, characterized in that, The second sub-transformation relationship is determined based on the joint angles of the labeling robot.

8. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-4.

9. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-4.

Images