A method for calculating the application position of a product and a self-positioning application method
Through the coordinated control of computer vision and robots, precise positioning and applying of different models of products is achieved, and the problems of low patch efficiency and high cost caused by random spatial position changes and model changes in the existing technology are solved, and the patching is improved Precision and efficiency.
Patent Information
- Application Number
- CN202411044951.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-01
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2044-08-01
AI Technical Summary
The existing industrial control technology cannot accurately locate and apply the product as the random spatial position changes, and cannot support automatic switching of positioning and applying different product models, resulting in low patch efficiency and high cost, and cannot be suitable for production lines of different models of products. .
Using a product's patch position calculation method, by obtaining the distance measuring equipment and camera data, calculate the patch coordinates and angle offsets of the products to be patched in the world coordinate system, and combine image processing and robot motion control to achieve different Accurate positioning and automatic patching of models.
It realizes accurate positioning and precise patching of various models of products, improves patching efficiency and accuracy, supports rapid production line deployment and automatic replacement of multiple models of products, and reduces production costs.
Smart Images

Figure CN118578401B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of industrial control technology, and in particular to a method for calculating a pasting position of a product applied to a robot pasting system and a self-positioning pasting method. Background Art
[0002] Household appliances refer to various types of electrical equipment that provide convenience and comfort for people's lives, including refrigerators, washing machines, air conditioners, televisions, water heaters, range hoods, disinfection cabinets, rice cookers, etc. There are many types of household appliances that can meet people's needs in daily life, work, entertainment, study, etc. With the continuous development of science and technology, household appliances are also constantly upgraded and improved. In the future, household appliances will be more intelligent and user-friendly.
[0003] In the production process of home appliances, labels, nameplates or warning stickers need to be applied to the appliances. In the process of applying printed materials such as labels, nameplates or warning stickers, in the case of some products whose spatial positions are strictly fixed and very standardized, they can be automatically applied by machines, but this condition is very harsh, and it is difficult to meet these requirements in the case of assembly line operation. This will result in the inability to produce mixed models on the production line. When the product model shape varies greatly or the placement position is very different, it will cause positioning failure. Summary of the invention
[0004] The technical problems to be solved by the present invention are: 1. The existing industrial control technology cannot achieve the problem of accurate positioning and pasting as the random spatial position of the product changes; 2. The existing industrial control technology cannot support automatic switching positioning and pasting of products of different product models, cannot achieve rapid deployment of production lines, and cannot be applied to production lines of products of different models, resulting in low pasting efficiency and high cost.
[0005] In order to solve the above problems, the technical solution adopted by the present invention is: a method for calculating the pasting position of a product, comprising the following steps: S1, obtaining the distance measurement value laserX between the distance measurement device and the surface to be pasted of the product to be pasted when the distance measurement device moves to the distance measurement position; S2, calculating the coordinates of the photographing point (pcX, pcY, pcZ) of the product to be pasted in the world coordinate system according to the distance measurement value laserX, S3, obtaining the image data of the image of the product to be pasted taken when the camera moves to the photographing point coordinates (pcX, pcY, pcZ); S4, calculating the corner point coordinates (ptY, ptZ, A) of the product to be pasted in the product image coordinate system according to the image data; S5, calculating the pasting coordinates (tX, tY, tZ) and the angle offset tA of the product to be pasted in the world coordinate system based on the corner point coordinates (ptY, ptZ, A), thereby obtaining the pasting position (tX, tY, tZ, tA). This technical solution can solve the problem of correcting the pasting position due to random position changes of products of the same type.
[0006] Furthermore, the calculation steps of the corner point coordinates (ptY, ptZ, A) of the product to be pasted in S4 are as follows: (1) threshold segmentation of the image data obtains the target area of the corner point of the product to be pasted, and a closing operation is performed to segment the effective area of the target corner point; (2) convolution of the effective area is performed to obtain the edge, the edge is segmented, the collinear edges are merged, the length and angle are screened, and the straight line is fitted to finally obtain the edge straight line in the Z-axis direction and the edge straight line in the Y-axis direction of the image coordinate system of the product to be pasted; the edge image position ptZ in the Z-axis direction, the edge image position ptY in the Y-axis direction, and the angle A of the edge image in the Y-axis direction of the product to be pasted are calculated to obtain the corner point coordinates (ptY, ptZ, A). This technical solution can accurately calculate the corner point coordinates.
[0007] Furthermore, before step S1, step S0 is also included, obtaining and setting system parameters: the distance measurement value slaserX between the distance measuring device and the surface to be pasted of the standard product when the camera object distance is the standard object distance of the camera during the standard product pasting process; the edge image position pZ of the corner point of the standard product in the image coordinate system Z-axis direction, the edge image position pY of the image coordinate system Y-axis direction, and the coordinates of the photographing point (spcX, spcY, spcZ) of the standard product when the corner point of the standard product reaches the calibration corner point position; the manipulator pasting position (stX, stY, stZ) when the corner point of the pasted material coincides with the corner point of the standard product. Obtaining and setting system parameters facilitates the algorithm to retrieve parameter data and achieve the purpose of automatically calculating and correcting the pasting position.
[0008] A method for calculating the pasting position of a product, comprising the following steps: S1, calculating the offset of the distance measurement position in the X-axis direction of the world coordinate system based on the parameters of the product to be pasted and the standard product, and obtaining the distance measurement value laserX between the distance measurement device and the pasting surface of the product to be pasted when the distance measurement device moves to the distance measurement position according to the offset; the parameters of the standard product at least include height, width and thickness; the parameters of the product to be pasted at least include height, width and thickness; S2, calculating the photographic value laserX of the product to be pasted in the world coordinate system based on the parameters of the product to be pasted, the parameters of the standard product and the distance measurement value laserX Point coordinates (pcX, pcY, pcZ); S3, obtain the image data of the image of the product to be pasted taken when the camera moves to the shooting point coordinates (pcX, pcY, pcZ); S4, calculate the corner point coordinates (ptY, ptZ, A) of the product to be pasted in the product image coordinate system according to the image data; S5, calculate the application coordinates (tX, tY, tZ) and angle offset tA of the product to be pasted in the world coordinates based on the corner point coordinates (ptY, ptZ, A) and the parameters of the product to be pasted, so as to obtain the application position (tX, tY, tZ, tA). On the one hand, the product to be pasted and the standard product will have different height, width and thickness parameters, resulting in a different application position from the standard product; on the other hand, the product to be pasted will also change at a random position, resulting in an inaccurate application position. This technical solution solves the problem of correction calculation of the application position of products of different models.
[0009] Furthermore, in S1, the offset of the distance measurement position in the X-axis direction of the world coordinate system is calculated according to jF*xF*(L-sL), wherein jF is the clamping and pushing direction value, and xF is the X-axis direction value of the manipulator.
[0010] Furthermore, in S2, the coordinates of the photographing point (pcX, pcY, pcZ) of the product to be pasted in the world coordinate system are calculated based on the parameters of the product to be pasted, the parameters of the standard product and the distance measurement value laserX; the calculation formula is as follows:
[0011] pcX=spcX+jF*xF*(L-sL)+(laserX-slaserX)
[0012] pcY=spcY+lF*yF*(W-sW)
[0013] pcZ=spcZ+zF*(H-sH)
[0014] In the formula, slaserX is the distance measurement value between the distance measuring device and the surface to be pasted of the standard product during the standard product pasting process, spcX, spcY and spcZ are the coordinates of the photographing point in the world coordinate system of the standard product during the standard product pasting process, sH, sW, sL are the height, width and thickness of the standard product, H, W, L are the height, width and thickness of the product to be pasted, xF, yF, zF are the X, Y, Z axis direction values of the robot, lF is the line flow direction value, and jF is the clamping push direction value.
[0015] Furthermore, the calculation formulas of the application coordinates (tX, tY, tZ) and the angle offset tA of the product to be applied in the world coordinates in S5 are as follows:
[0016] tX=stX+(pcX-spcX)
[0017] tY=stY+(pcY-spcY)+G(pY-ptY)+offY
[0018] tZ=stZ+(pcZ-spcZ)+G(pZ-ptZ)+offZ
[0019] tA=A
[0020] In the formula, spcX, spcY and spcZ are the coordinates of the photographing point of the standard product in the world coordinate system during the standard product pasting process, G is the operator for converting the image coordinate system to the world coordinate system in the calibration file; stX, stY and stZ are the manipulator pasting positions (stX, stY, stZ) when the corner points of the pasted material coincide with the corner points of the standard product during the standard product pasting process, pZ and pY are the edge image positions of the corner points of the standard product in the Z-axis direction of the image coordinate system and the edge image positions of the Y-axis direction of the image coordinate system when the corner points of the standard product reach the calibration corner point position during the standard product pasting process, and offZ and offY are the pasting margins in the Z-axis direction of the world coordinate system and the pasting margins in the Y-axis direction of the world coordinate system in the parameters of the product to be pasted. Different types of products require different offZ and offY, which can improve the applicability of the pasting method.
[0021] Furthermore, the calculation formulas of the application coordinates (tX, tY, tZ) and the angle offset tA of the product to be applied in the world coordinates in S5 are as follows:
[0022] tX=stX+(pcX-spcX)
[0023] tY=stY+(pcY-spcY)+G(pY-ptY)+offY+eY
[0024] tZ=stZ+(pcZ-spcZ)+G(pZ-ptZ)+offZ+eZ
[0025] tA=A
[0026] In the formula, eZ and eY are the additional height compensation value and additional width compensation value in the parameters of the product to be pasted, respectively.
[0027] Furthermore, before step S1, the following step S0 is also included, obtaining and setting system parameters: the height sH, width sW and thickness sL of the standard product, the distance measurement value slaserX between the distance measuring device and the surface to be pasted of the standard product when the camera object distance is the standard object distance of the camera during the standard product pasting process; the edge image position pZ of the corner point of the standard product in the image coordinate system Z axis direction, the edge image position pY of the image coordinate system Y axis direction and the coordinates of the photographing point (spcX, spcY, spcZ) of the standard product when the corner point of the standard product reaches the calibration corner point position; the manipulator pasting position (stX, stY, stZ) when the corner point of the pasted material coincides with the corner point of the standard product; the manipulator X, Y, Z axis direction values xF, yF, zF, the linear flow direction value lF, and the clamping and pushing direction value jF. Obtaining and setting system parameters facilitates the algorithm to retrieve parameter data and achieve the purpose of automatically calculating and correcting the pasting position.
[0028] Furthermore, between step S0 and step S1, step S01 is also included, obtaining parameters of the product to be pasted, wherein the parameters of the product to be pasted include the height H, width W, thickness L, pasting margin offZ in the Z-axis direction of the world coordinate system, pasting margin offY in the Y-axis direction of the world coordinate system, additional height compensation value eZ and additional width compensation value eY of the product to be pasted.
[0029] A self-positioning pasting method for a product, based on the pasting position calculation method of the above product, also includes step S6, controlling the manipulator to move to the pasting position (tX, tY, tZ, tA) to apply the material to the product to be pasted. By obtaining the parameters of the product to be pasted, the product model of the production line can be adjusted, the model can be automatically identified, and the model can be quickly changed for pasting, which can support the mixed production of different models of equipment on the production line.
[0030] Furthermore, in S0, the system parameters are obtained and set by means of standard product pasting calibration: the camera calibration obtains the calibration file, the height sH, width sW and thickness sL of the standard product are measured, the manipulator is moved so that the camera object distance is just the standard object distance of the camera, and the distance measurement value slaserX measured by the test equipment at this time is recorded. The Y-axis and Z-axis coordinates of the manipulator world coordinates are manually changed so that the corner points of the standard product coincide with the auxiliary lines of the calibration corner points in the calibration page, and the edge positions pZ and p of the corner points of the standard product in the image coordinate system in the Z-axis direction and the Y-axis direction of the image coordinate system are recorded. The camera shooting point position (spcX, spcY, spcZ) is recorded when taking pictures. The manipulator moves to the pasting point so that the corner points of the pasted material coincide with the corner points of the standard product, and the pasting position of the manipulator during pasting (stX, stY, stZ) is recorded. The system parameters can be quickly and accurately obtained and set by means of calibration.
[0031] Further, S01, the model data and parameters of the product to be pasted are stored in a database in advance, and the corresponding parameters are retrieved from the database according to the model data of the product to be pasted. Quickly reading and retrieving parameters can improve the pasting efficiency.
[0032] Compared with the prior art, the invention has the following beneficial effects: the invention can realize precise positioning and precise pasting by visually positioning various types and models of products with corner points and using algorithms to correct pasting angles and pasting positions. It can realize the positioning of the spatial position of the product, solve the shortcomings of the pasting process that cannot realize random spatial positions, and improve the pasting accuracy. It can realize rapid deployment of equipment, automatic model change, and automatic correction, which is convenient for the random switching of multiple models of products. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0034] Figure 1 This is a flow chart of the method for calculating the application position of the first product of the present invention.
[0035] Figure 2 This is a flow chart of the method for calculating the application position of the second product of the present invention.
[0036] Figure 3 This is a diagram of the corner point calibration interface used in the standard product application calibration of the present invention. DETAILED DESCRIPTION
[0037] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. The embodiments of the present invention are provided for the purpose of illustration and description, and are not intended to be exhaustive or to limit the present invention to the disclosed forms. Many modifications and variations will be apparent to those of ordinary skill in the art. The embodiments are selected and described in order to better illustrate the principles and practical applications of the present invention, and to enable those of ordinary skill in the art to understand the present invention and thereby design various embodiments with various modifications suitable for specific uses.
[0038] In the existing process of applying printed materials such as labels, nameplates or warning stickers, the random spatial position of the products on the production line will change, and the existing application system cannot accurately locate the application position; it cannot support automatic switching positioning and application of products of different product models, cannot achieve rapid deployment of production lines, and cannot be applied to production lines of different product models, resulting in low application efficiency and high cost.
[0039] To this end, the present invention provides several specific embodiments. Before elaborating on the method of the present invention in detail, it should be noted that the height, width and thickness of the product to be pasted mentioned in this article correspond to the Z-axis direction, Y-axis direction and X-axis direction in the system world coordinate system and the image coordinate system.
[0040] Embodiment 1: Figure 1 A method for calculating the application position of a product is used in a robot application system. The robot application system is mainly configured with a computer, a camera, a distance measuring device and a robot. Generally, the camera and the distance measuring device are installed on the robot so that they can be moved. The distance measuring device is preferably a laser distance measuring device, but is not limited to this structure. The hardware structure of the robot application system is not a protection point of the present invention and will not be elaborated here.
[0041] The method for calculating the pasting position of a product in this embodiment is aimed at products of the same model with corner points, that is, the product to be pasted and the standard product are the same model products, so as to solve the problem of correcting the pasting position to achieve accurate pasting when the random position of the same model product on the production line changes, and specifically includes the following steps:
[0042] S1. Obtain the distance value laserX between the distance measuring device and the surface to be pasted of the product to be pasted when the distance measuring device moves to the distance measuring position; S2. Calculate the coordinates of the photographing point (pcX, pcY, pcZ) of the product to be pasted in the world coordinate system according to the distance measuring value laserX; S3. Obtain the image data of the image of the product to be pasted taken when the camera moves to the photographing point coordinates (pcX, pcY, pcZ); preferably, lighting is used during shooting to improve the image quality; S4. Calculate the corner point coordinates (ptY, ptZ, A) of the product to be pasted in the product image coordinate system according to the image data; S5. Calculate the application coordinates (tX, tY, tZ) and the angle offset tA of the product to be pasted in the world coordinate system based on the corner point coordinates (ptY, ptZ, A), thereby obtaining the application position (tX, tY, tZ, tA).
[0043] Specifically, the steps for calculating the corner point coordinates (ptY, ptZ, A) of the product to be pasted in S4 are as follows: (1) threshold segmenting the image data to obtain the target area of the corner point of the product to be pasted, the target area refers to the scope area covering the corner point, and the segmentation is intended to narrow the scope of image processing, implement a closed operation, and segment the effective area of the target corner point; 2) convolve the effective area to obtain the edge, segment the edge, merge the collinear edges, perform length and angle screening, fit the straight line, and finally obtain the edge straight line in the Z-axis direction and the edge straight line in the Y-axis direction of the image coordinate system of the product to be pasted; calculate the edge image position ptZ in the Z-axis direction, the edge image position ptY in the Y-axis direction, and the angle A of the edge image in the Y-axis direction of the product to be pasted, and obtain the corner point coordinates (ptY, ptZ, A).
[0044] Specifically, since the product to be pasted and the standard product are of the same model, there is no offset in the distance measurement position of the distance measurement device, and the distance measurement value is set to measure the distance measurement value laserX at the initial distance measurement position. Therefore, the coordinates of the photographing point of the product to be pasted in the world coordinate system (pcX, pcY, pcZ) are calculated as follows:
[0045] pcX=spcX+(laserX-slaserX)
[0046] pcY=spcY
[0047] pcZ=spcZ
[0048] The application position (tX, tY, tZ, tA) is calculated as follows:
[0049] tX=stX+(pcX-spcX)
[0050] tY=stY+(pcY-spcY)+G(pY-ptY)
[0051] tZ=stZ+(pcZ-spcZ)+G(pZ-ptZ)
[0052] tA=A
[0053] In the formula, slaserX is the distance measurement value between the distance measuring device and the surface to be pasted of the standard product during the standard product pasting process, spcX, spcY and spcZ are respectively the coordinates of the photographing point of the standard product in the world coordinate system during the standard product pasting process, G is an operator for converting the image coordinate system into the world coordinate system in the calibration file, and the calibration file is obtained through camera calibration, which is the prior art; stX, stY and stZ are respectively the manipulator pasting positions (stX, stY, stZ) when the corner point of the pasted material coincides with the corner point of the standard product during the standard product pasting process, pZ and pY are respectively the edge image positions of the corner points of the standard product in the Z-axis direction of the image coordinate system and the edge image positions of the corner points of the standard product in the Y-axis direction of the image coordinate system when the corner points of the standard product reach the calibration corner point positions during the standard product pasting process.
[0054] Preferably, the calculation formulas of the application coordinates (tX, tY, tZ) and the angle offset tA of the product to be applied in the world coordinates are as follows:
[0055] tX=stX+(pcX-spcX)
[0056] tY=stY+(pcY-spcY)+G(pY-ptY)+eY
[0057] tZ=stZ+(pcZ-spcZ)+G(pZ-ptZ)+eZ
[0058] tA=A
[0059] Wherein, eY is the extra width compensation value, and eZ is the extra height compensation value.
[0060] The introduction of additional compensation values improves the stability of the technical solution, and can quickly adjust and output correct results when there are slight changes in hardware parameters, reduce the adverse effects of changes in system hardware parameters on the precise repair of the application position, and improve the accuracy of the application position calculation.
[0061] Preferably, before step S1, step S0 is also included, obtaining and setting system parameters: the distance measurement value slaserX between the distance measuring device and the surface to be pasted of the standard product when the camera object distance is the standard object distance of the camera during the standard product pasting process; the edge image position pZ of the corner point of the standard product in the Z-axis direction of the image coordinate system, the edge image position pY in the Y-axis direction of the image coordinate system, and the coordinates of the photographing point (spcX, spcY, spcZ) of the standard product when the corner point of the standard product reaches the calibration corner point position; the manipulator pasting position (stX, stY, stZ) when the corner point of the pasted material coincides with the corner point of the standard product.
[0062] This embodiment can solve the problem that the same model of products cannot be accurately positioned and applied due to random spatial position changes, and has high application accuracy.
[0063] Embodiment 2: Figure 2 A method for calculating the application position of a product is used in a robot application system. The robot application system is mainly configured with a computer, a camera, a distance measuring device and a robot. Generally, the camera and the distance measuring device are installed on the robot so that they can be moved. The distance measuring device is preferably a laser distance measuring device, but is not limited to this structure. The hardware structure of the robot application system is not a protection point of the present invention and will not be elaborated here.
[0064] This embodiment is aimed at products of different models with corner points, that is, the product to be pasted and the standard product are of the same type of product but of different models, so as to solve the problem of correcting the pasting position to achieve accurate pasting when the positions of products of different models change, and specifically includes the following steps:
[0065] S1. Calculate the offset of the distance measuring position in the X-axis direction of the world coordinate system based on the parameters of the product to be pasted and the standard product, and obtain the distance measuring value laserX between the distance measuring device and the surface to be pasted of the product to be pasted when the distance measuring device moves to the distance measuring position according to the offset; the parameters of the standard product at least include the height sH, width sW and thickness sL of the standard model product, and the parameters of the product to be pasted at least include the height H, width W, thickness L, the pasting margin offZ in the Z-axis direction of the world coordinate system, and the pasting margin offY in the Y-axis direction of the world coordinate system. The pasting margin offZ and the pasting margin offY are both pasting index values required by the production line. Different models of products can set different pasting margins offZ and pasting margins offY according to the manufacturer's requirements.
[0066] The offset of the distance measuring position in the X-axis direction of the world coordinate system is calculated according to jF*xF*(L-sL), where jF is the clamping push direction value, which takes a value of 0 or 1 and can be set in the system parameters according to the situation; xF is the X-axis direction value of the manipulator, which takes a value of 1 or -1 and can be set in the system parameters according to the situation; when the offset is not zero, the distance measuring device moves from the initial distance measuring position according to the offset and measures the distance measuring value laserX between it and the surface to be laminated of the product to be laminated.
[0067] S2. Calculate the coordinates of the photographing point (pcX, pcY, pcZ) of the product to be pasted in the world coordinate system based on the parameters of the product to be pasted, the parameters of the standard product and the distance measurement value laserX. The calculation formula is as follows:
[0068] pcX=spcX+jF*xF*(L-sL)+(laserX-slaserX)
[0069] pcY=spcY+lF*yF*(W-sW)
[0070] pcZ=spcZ+zF*(H-sH)
[0071] In the formula, slaserX is the distance value between the distance measuring device and the surface to be pasted of the standard product during the standard product pasting process, spcX, spcY and spcZ are the coordinates of the photographing point in the world coordinate system of the standard product during the standard product pasting process, sH, sW, sL are the height, width and thickness of the standard product, H, W, L are the height, width and thickness of the product to be pasted, xF, yF, zF are the X, Y and Z axis direction values of the manipulator, which are 1 or -1 and can be set in the system parameters according to the situation, lF is the line flow direction value, which is 0 or 1 and can be set in the system parameters according to the situation, jF is the clamping push direction value, which is 0 or 1 and can be set in the system parameters according to the situation.
[0072] S3, obtain the image data of the image of the product to be pasted when the camera moves to the coordinates of the shooting point (pcX, pcY, pcZ); preferably, lighting is used during shooting to improve the image quality. S4, calculate the corner point coordinates (ptY, ptZ, A) of the product to be pasted in the product image coordinate system according to the image data; the corner point coordinates (ptY, ptZ, A) of the product to be pasted are calculated in the following steps: (1) threshold segmentation of the image data is performed to obtain the target area of the corner point of the product to be pasted, the target area refers to the scope area covering the corner point, the segmentation is intended to narrow the scope of image processing, and a closed operation is performed to segment the effective area of the target corner point; (2) convolve the effective area to obtain the edge, segment the edge, merge the collinear edges, perform length and angle screening, fit the straight line, and finally obtain the edge straight line in the Z-axis direction and the edge straight line in the Y-axis direction of the image coordinate system of the product to be pasted; calculate the edge image position ptZ in the Z-axis direction, the edge image position ptY in the Y-axis direction and the angle A of the edge image in the Y-axis direction of the product to be pasted, and obtain the corner point coordinates (ptY, ptZ, A).
[0073] S5. Calculate the pasting coordinates (tX, tY, tZ) and angle offset tA of the product to be pasted in the world coordinates based on the corner point coordinates (ptY, ptZ, A) and the parameters of the product to be pasted, so as to obtain the pasting position (tX, tY, tZ, tA). The calculation formula is as follows:
[0074] tX=stX+(pcX-spcX)
[0075] tY=stY+(pcY-spcY)+G(pY-ptY)+offY
[0076] tZ=stZ+(pcZ-spcZ)+G(pZ-ptZ)+offZ
[0077] tA=A
[0078] In the formula, spcX, spcY and spcZ are the coordinates of the photographing points of the standard product in the world coordinate system during the standard product pasting process, G is an operator for converting the image coordinate system into the world coordinate system in the calibration file, and the calibration file is obtained through camera calibration, which is the prior art; stX, stY and stZ are the robot pasting positions (stX, stY, stZ) when the corner points of the pasted material coincide with the corner points of the standard product during the standard product pasting process, pZ and pY are the edge image positions of the corner points of the standard product in the Z-axis direction of the image coordinate system and the Y-axis direction of the image coordinate system when the corner points of the standard product reach the calibration corner point positions during the standard product pasting process, and offZ and offY are the pasting margins in the Z-axis direction of the world coordinate system and the Y-axis direction of the world coordinate system in the parameters of the product to be pasted, respectively.
[0079] Preferably, the parameters of the product to be pasted further include: an additional height compensation value eZ and an additional width compensation value eY; the calculation formula of the pasting coordinates (tX, tY, tZ) and the angle offset tA of the product to be pasted in the world coordinates in S5 is as follows:
[0080] tX=stX+(pcX-spcX)
[0081] tY=stY+(pcY-spcY)+G(pY-ptY)+offY+eY
[0082] tZ=stZ+(pcZ-spcZ)+G(pZ-ptZ)+offZ+eZ
[0083] tA=A
[0084] In the formula, spcX, spcY and spcZ are the coordinates of the photographing point of the standard product in the world coordinate system during the standard product pasting process, G is the operator for converting the image coordinate system into the world coordinate system by the calibration file, and the calibration file is obtained by camera calibration, which is the prior art; stX, stY and stZ are the manipulator pasting positions (stX, stY, stZ) when the corner points of the pasted material coincide with the corner points of the standard product during the standard product pasting process, pZ and pY are the edge image positions of the corner points of the standard product in the Z-axis direction of the image coordinate system and the edge image positions of the Y-axis direction of the image coordinate system when the corner points of the standard product reach the calibration corner point positions during the standard product pasting process, and offZ and offY are the pasting margins in the Z-axis direction of the world coordinate system and the pasting margins in the Y-axis direction of the world coordinate system in the parameters of the product to be pasted. The introduction of additional compensation values improves the stability of the technical solution, and can quickly adjust and output correct results when the hardware parameters change slightly, reduce the adverse effects of changes in hardware parameters on the precise repair of the pasting position, and improve the accuracy of the pasting position calculation.
[0085] Before step S1, the following step S0 is also included, obtaining and setting system parameters: height sH, width sW and thickness sL of the standard product, distance measurement value slaserX between the distance measuring device and the surface to be pasted of the standard product when the camera object distance is the standard object distance of the camera during the standard product pasting process; edge image position pZ of the corner point of the standard product in the Z-axis direction of the image coordinate system, edge image position pY of the Y-axis direction of the image coordinate system and photographing point coordinates (spcX, spcY, spcZ) of the standard product when the corner point of the standard product reaches the calibration corner point position; manipulator pasting position (stX, stY, stZ) when the corner point of the pasted material coincides with the corner point of the standard product; manipulator X, Y, Z axis direction values xF, yF, zF, line flow direction value lF, clamping and pushing direction value jF.
[0086] Between step S0 and step S1, step S01 is also included, obtaining the parameters of the product to be pasted, the parameters of the product to be pasted include the height H, width W, thickness L, pasting margin offZ in the Z-axis direction of the world coordinate system, pasting margin offY in the Y-axis direction of the world coordinate system, additional height compensation value eZ and additional width compensation value eY. The parameters of the product to be pasted can be obtained once for each product in real time to ensure that the pasting system can correctly identify the parameters of the current product to be pasted, accurately correct the pasting position, solve the problem of inaccurate pasting due to production line change, and achieve the purpose of automatic switching pasting.
[0087] This embodiment can achieve the purpose of automatically switching and pasting different types of products on a production line, solve the problem of being unable to accurately position and paste due to random spatial position changes of different types of products, and has high pasting accuracy and efficiency, thereby reducing production costs.
[0088] Embodiment 3: A method for self-positioning application of a product, based on the method for calculating the application position of the product in embodiment 1, further comprising step S6, controlling the robot arm to move to the application position (tX, tY, tZ, tA) to apply material to the product to be applied.
[0089] Preferably, in S0, the system parameters are obtained and set by means of standard product application calibration: the camera calibration obtains the calibration file, moves the manipulator so that the camera object distance is just the standard object distance of the camera, and records the distance measurement value slaserX measured by the test equipment at this time, such as Figure 3 , manually change the Y-axis and Z-axis coordinates of the robot world coordinates, so that the corner points of the standard product coincide with the auxiliary lines of the calibration corner points in the calibration page, record the edge position pZ of the corner points of the standard product in the image coordinate system in the Z-axis direction, the edge position pY of the image coordinate system in the Y-axis direction, record the camera's shooting point position (spcX, spcY, spcZ) when taking pictures, the robot moves to the application point, so that the corner points of the application material coincide with the corner points of the standard product, and record the application position of the robot during application (stX, stY, stZ). The standard product application calibration method can quickly and accurately obtain and set system parameters to improve labor efficiency.
[0090] Embodiment 4: On the basis of embodiment 2, a self-positioning pasting method for a product is provided, based on the pasting position calculation method of the product in embodiment 1, and further includes step S6, controlling the robot arm to move to the pasting position (tX, tY, tZ, tA) to apply material to the product to be pasted.
[0091] Preferably, in S0, the system parameters are obtained and set by means of standard product pasting calibration: the camera calibration obtains the calibration file, the height sH, width sW and thickness sL of the standard product are measured, the manipulator is moved so that the camera object distance is just the standard object distance of the camera, and the distance measurement value slaserX measured by the test equipment at this time is recorded, such as Figure 3 , manually change the Y-axis and Z-axis coordinates of the robot world coordinates, so that the corner points of the standard product coincide with the auxiliary lines of the calibration corner points in the calibration page, record the edge position pZ of the corner points of the standard product in the image coordinate system in the Z-axis direction, the edge position pY of the image coordinate system in the Y-axis direction, record the camera's shooting point position (spcX, spcY, spcZ) when taking pictures, the robot moves to the application point, so that the corner points of the application material coincide with the corner points of the standard product, and record the application position of the robot during application (stX, stY, stZ). The standard product application calibration method can quickly and accurately obtain and set system parameters to improve labor efficiency.
[0092] The model data and parameters of the product to be pasted in S01 are stored in the database in advance, and the corresponding parameters are retrieved from the database according to the model data of the product to be pasted. The specific product model data is identified by scanning the barcode of the product to obtain the model data. This technical feature is to recognize the model data of the product to be pasted and call the parameters, so as to achieve the purpose of fast switching of pasting.
[0093] This embodiment can solve the problem of obtaining and setting system parameters, and can quickly obtain system parameters in the process of standard product pasting. Once the system parameters are set, pasting can be automatically switched when different models of products are produced on a production line, which can ensure that the robot pasting system only needs to be calibrated once from installation to use on a production line, thereby achieving the purpose of rapid deployment of the production line. The pasting material in this article can be a label, nameplate, warning sign or protective film.
[0094] The present invention achieves precise positioning and precise pasting by visually positioning various types and models of products with corner points, coordinating with lighting, and using algorithms to correct pasting angles and pasting positions. It realizes the positioning of the spatial position of the product, solves the shortcomings of the pasting process that cannot achieve random spatial positions, and improves the pasting accuracy. It realizes rapid deployment of equipment, automatic model change, and automatic correction, which is convenient for the random switching of multiple models of products.
[0095] According to the present embodiment, an electronic device, a readable storage medium and a computer program product are also provided. The electronic device is intended to represent various forms of digital computers and is not intended to limit the implementation of the present disclosure described and / or claimed herein.
[0096] An electronic device comprises: at least one processor, and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute a method for calculating a location of application of the product and a method for self-positioning application.
[0097] The computer instructions are computer programs that implement a certain function. The program code for implementing the method for calculating the application position of the product and the method for self-positioning application can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor, implements the functions / operations specified in the flow chart and / or block diagram. The program code can be executed entirely on the machine, partially on the machine, partially on the machine as an independent software package and partially on a remote machine, or entirely on a remote machine or server.
[0098] The processor can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) or a computer program loaded from a storage unit to a random access memory (RAM). The processor, ROM, and RAM are connected to each other through a bus.
[0099] In this embodiment, a non-transitory computer-readable storage medium stores computer instructions, wherein the computer instructions are used to enable the computer to execute the product application position calculation method and the self-positioning application method.
[0100] The machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium.
[0101] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be performed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in the present invention can be achieved, and this document does not limit this.
[0102] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the principles of the present disclosure shall be included in the protection scope of the present disclosure.
Claims
1. A method for calculating the application position of a product, used in a robot application system, characterized in that: The steps include: S1. Calculate the offset of the distance measurement position in the X-axis direction of the world coordinate system based on the parameters of the product to be pasted and the standard product, and obtain the distance measurement value laserX between the distance measurement device and the surface to be pasted of the product to be pasted when the distance measurement device moves to the distance measurement position according to the offset; the parameters of the standard product at least include the height sH, width sW and thickness sL; the parameters of the product to be pasted at least include the height H, width W and thickness L; S2. Calculate the coordinates of the photographing point (pcX, pcY, pcZ) of the product to be pasted in the world coordinate system based on the parameters of the product to be pasted, the parameters of the standard product and the distance measurement value laserX. The calculation formula is as follows: pcX=spcX+jF*xF*(L-sL)+(laserX-slaserX) pcY=spcY+lF*yF*(W-sW) pcZ=spcZ+zF*(H-sH) In the formula, slaserX is the distance value between the distance measuring device and the surface to be pasted of the standard product during the standard product pasting process, spcX, spcY and spcZ are the coordinates of the photographing point in the world coordinate system of the standard product during the standard product pasting process, xF, yF, zF are the X, Y and Z axis direction values of the manipulator respectively, lF is the flow direction value of the line body, and jF is the clamping and pushing direction value; S3, obtaining image data of the image of the product to be pasted taken when the camera moves to the coordinates of the shooting point (pcX, pcY, pcZ); S4. Calculate the corner point coordinates (ptY, ptZ, A) of the product to be pasted in the product image coordinate system according to the image data; S5. Calculate the pasting coordinates (tX, tY, tZ) and the angle offset tA of the product to be pasted in the world coordinates based on the corner point coordinates (ptY, ptZ, A) and the parameters of the product to be pasted, so as to obtain the pasting position (tX, tY, tZ, tA). The calculation formula is as follows: tX=stX+(pcX-spcX) tY=stY+(pcY-spcY)+G(pY-ptY)+offY tZ=stZ+(pcZ-spcZ)+G(pZ-ptZ)+offZ tA=A In the formula, G is the operator for converting the image coordinate system of the calibration file into the world coordinate system; stX, stY and stZ are the manipulator pasting positions (stX, stY, stZ) when the corner points of the pasted material coincide with the corner points of the standard product during the pasting process of the standard product, respectively; pZ and pY are the edge image positions of the corner points of the standard product in the Z-axis direction of the image coordinate system and the edge image positions of the Y-axis direction of the image coordinate system when the corner points of the standard product reach the calibration corner point positions during the pasting process of the standard product, respectively; offZ and offY are the pasting margins in the Z-axis direction of the world coordinate system and the pasting margins in the Y-axis direction of the world coordinate system in the parameters of the product to be pasted, respectively.
2. A method for calculating the application position of a product, used in a robot application system, characterized in that: The steps include: S1. Calculate the offset of the distance measurement position in the X-axis direction of the world coordinate system based on the parameters of the product to be pasted and the standard product, and obtain the distance measurement value laserX between the distance measurement device and the surface to be pasted of the product to be pasted when the distance measurement device moves to the distance measurement position according to the offset; the parameters of the standard product at least include the height sH, width sW and thickness sL; the parameters of the product to be pasted at least include the height H, width W and thickness L; S2. Calculate the coordinates of the photographing point (pcX, pcY, pcZ) of the product to be pasted in the world coordinate system based on the parameters of the product to be pasted, the parameters of the standard product and the distance measurement value laserX. The calculation formula is as follows: pcX=spcX+jF*xF*(L-sL)+(laserX-slaserX) pcY=spcY+lF*yF*(W-sW) pcZ=spcZ+zF*(H-sH) In the formula, slaserX is the distance value between the distance measuring device and the surface to be pasted of the standard product during the standard product pasting process, spcX, spcY and spcZ are the coordinates of the photographing point in the world coordinate system of the standard product during the standard product pasting process, xF, yF, zF are the X, Y and Z axis direction values of the manipulator respectively, lF is the flow direction value of the line body, and jF is the clamping and pushing direction value; S3, obtaining image data of the image of the product to be pasted taken when the camera moves to the coordinates of the shooting point (pcX, pcY, pcZ); S4. Calculate the corner point coordinates (ptY, ptZ, A) of the product to be pasted in the product image coordinate system according to the image data; S5. Calculate the pasting coordinates (tX, tY, tZ) and the angle offset tA of the product to be pasted in the world coordinates based on the corner point coordinates (ptY, ptZ, A) and the parameters of the product to be pasted, so as to obtain the pasting position (tX, tY, tZ, tA). The calculation formula is as follows: tX=stX+(pcX-spcX) tY=stY+(pcY-spcY)+G(pY-ptY)+offY+eY tZ=stZ+(pcZ-spcZ)+G(pZ-ptZ)+offZ+eZ tA=A In the formula, G is the operator for converting the image coordinate system of the calibration file into the world coordinate system; stX, stY and stZ are the robot arm pasting positions (stX, stY, stZ) when the corner points of the pasted material coincide with the corner points of the standard product during the pasting process of the standard product, pZ and pY are the edge image positions of the corner points of the standard product in the Z-axis direction of the image coordinate system and the edge image positions of the Y-axis direction of the image coordinate system when the corner points of the standard product reach the calibration corner point positions during the pasting process of the standard product, offZ and offY are the pasting margins in the Z-axis direction of the world coordinate system and the pasting margins in the Y-axis direction of the world coordinate system in the parameters of the product to be pasted, respectively; eZ and eY are the additional height compensation value and the additional width compensation value in the parameters of the product to be pasted, respectively.
3. A method for calculating the application position of a product as claimed in claim 1 or 2, characterized in that: In S1, the offset of the world coordinate system X-axis direction of the ranging position is calculated according to jF*xF*(L-sL).
4. A method for calculating the application position of a product as claimed in claim 1 or 2, characterized in that: The steps for calculating the corner point coordinates (ptY, ptZ, A) of the product to be pasted in S4 are as follows; (1) Segment the image data by threshold to obtain the target area of the corner points of the product to be pasted, perform a closing operation, and segment the effective area of the target corner points; (2) Perform convolution on the effective area to obtain the edge, segment the edge, merge the collinear edges, perform length and angle screening, fit the straight line, and finally obtain the edge straight line in the Z-axis direction and the edge straight line in the Y-axis direction of the image coordinate system of the product to be pasted; calculate the edge image position ptZ in the Z-axis direction, the edge image position ptY in the Y-axis direction, and the angle A of the edge image in the Y-axis direction of the product to be pasted, and obtain the corner point coordinates (ptY, ptZ, A).
5. A method for calculating the application position of a product as claimed in claim 1 or 2, characterized in that: Before step S1, the following step S0 is also included, obtaining and setting system parameters: height sH, width sW and thickness sL of the standard product, distance measurement value slaserX between the distance measuring device and the surface to be pasted of the standard product when the camera object distance is the standard object distance of the camera during the standard product pasting process; edge image position pZ of the corner point of the standard product in the Z-axis direction of the image coordinate system, edge image position pY of the Y-axis direction of the image coordinate system and photographing point coordinates (spcX, spcY, spcZ) of the standard product when the corner point of the standard product reaches the calibration corner point position; manipulator pasting position (stX, stY, stZ) when the corner point of the pasted material coincides with the corner point of the standard product; manipulator X, Y, Z axis direction values xF, yF, zF, line flow direction value lF, clamping and pushing direction value jF.
6. A method for calculating the application position of a product as claimed in claim 1 or 2, characterized in that: Between step S0 and step S1, step S01 is also included, obtaining parameters of the product to be pasted, wherein the parameters of the product to be pasted include the height H, width W, thickness L, pasting margin offZ in the Z-axis direction of the world coordinate system, pasting margin offY in the Y-axis direction of the world coordinate system, additional height compensation value eZ and additional width compensation value eY of the product to be pasted.
7. A method for self-positioning application of a product, characterized in that: The method for calculating the application position of a product according to any one of claims 1 to 6 further includes step S6, controlling the robot to move to the application position (tX, tY, tZ, tA) to apply material to the product to be applied.
8. A method for self-positioning application of a product as claimed in claim 7, characterized in that: Step S0 also includes: obtaining and setting system parameters by means of standard product pasting calibration: obtaining a calibration file through camera calibration, measuring the height sH, width sW and thickness sL of the standard product, moving the manipulator so that the camera object distance is just the standard object distance of the camera, recording the distance measurement value slaserX measured by the test equipment at this time, manually changing the Y-axis and Z-axis coordinates of the manipulator world coordinates so that the corner points of the standard product coincide with the auxiliary lines of the calibration corner points in the calibration page, recording the edge position pZ of the corner points of the standard product in the image coordinate system in the Z-axis direction and the edge position pY of the image coordinate system in the Y-axis direction, recording the camera shooting point position (spcX, spcY, spcZ) when taking pictures, the manipulator moves to the pasting point so that the corner points of the pasted material coincide with the corner points of the standard product, and recording the pasting position (stX, stY, stZ) of the manipulator during pasting.
9. A self-positioning and applying method for a product as claimed in claim 8, characterized in that: Step S01 also includes: storing the model data and parameters of the product to be pasted in a database in advance, and retrieving corresponding parameters from the database according to the model data of the product to be pasted.
10. An electronic device, characterized in that include: At least one processor, and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method for calculating the application position of the product described in any one of claims 1 to 6 and the method for self-positioning application of the product described in any one of claims 7 to 9.
11. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to enable the computer to execute the method for calculating the application position of the product described in any one of claims 1 to 6 and the method for self-positioning application of the product described in any one of claims 7 to 9.
