A mechanical arm motion projection correction method based on double industrial cameras

Abstract

The application discloses a mechanical arm movement projection correction method based on double industrial cameras, takes a target object as a reference object, measures the length and width of the target object; the mechanical arm moves to the vicinity of the target object actually needed to be sucked through image recognition, establishes a coordinate system of the target object, calculates the coordinates of four corner points and a center point of the target object; two cameras respectively identify the pixel coordinates of the four corner points of the target object, calculate the projection perspective matrix of the two cameras; the two cameras identify the mark points of the end effector of the mechanical arm, acquire the pixel coordinates and mark as a starting point, control the mechanical arm to move downward by a distance to continue identifying the mark points of the end effector as an ending point; the starting point and the ending point are mapped to the coordinate system of the target object through the projection perspective matrix; the deviation of the mechanical arm from the target object is obtained through an error calculation module to realize correction. The application does not need to be calibrated, provides a secondary correction opportunity for the mechanical arm, has high calculation precision, and has practicality and applicability.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of machine vision of industrial robot arm, and relates to a robot arm motion projection correction method based on double industrial cameras. BACKGROUND

[0002] With the in-depth development of national industrial integration, informatization and intelligentization, the flexibility and repeatability precision of the robot arm have been greatly improved, and the multi-axis linkage robot arm is increasingly widely applied in industrial production and social services. In addition, machine vision develops rapidly, and in some dangerous environments unsuitable for manual operation or in the process of large-scale repetitive industrial production, machine vision is often used to replace manual vision. This greatly improves the production efficiency and the degree of automation.

[0003] In the process of using machine vision to detect processing and production, target recognition and positioning are essential. Traditional target recognition algorithms either fail to register due to defects in the algorithms themselves or have large deviations due to uncertain factors such as light and environment. In order to further improve the precision of the robot arm and reduce the risk of safety accidents, the distance deviation between the actual target and the end effector after movement needs to be corrected, and there is currently no control method for such errors. SUMMARY

[0004] The present application proposes a robot arm motion projection correction method based on double industrial cameras to overcome the above-mentioned problems existing in the prior art.

[0005] The technical scheme of the present application is a robot arm motion projection correction method based on double industrial cameras, comprising the following steps:

[0006] S1. Take a target object as a reference object and measure the length and width of the target object;

[0007] S2. The robot arm moves to the vicinity of the target object to be sucked by image recognition, establishes a coordinate system of the target object, and calculates the coordinates of the four corner points and the center point of the target object according to the measured values in S1;

[0008] S3. Two cameras are used to identify the pixel coordinates of the four corner points of the target object, and the projection perspective matrix of the two cameras is calculated together with the four corner points obtained in S2;

[0009] S4. The marker points of the end effector of the robot arm are identified by the two cameras, the pixel coordinates are obtained and recorded as the starting point, and then the robot arm is controlled to move downward by a certain distance, and the marker points of the end effector are identified again as the ending point;

[0010] S5. The starting point and the ending point are mapped into the target object coordinate system through the projection perspective matrix.

[0011] S6, obtaining the deviation of the mechanical arm from the target object by an error calculation module.

[0012] Preferably, the measurement of the target object in S1 is only required once, and subsequent measurement is not required.

[0013] Preferably, in S2, the right direction of the X-axis of the target object coordinate system is the positive direction, and the downward direction is the positive direction of the Y-axis; the unit of the coordinate system is centimeter, and the specific coordinates are consistent with the actual size of the object.

[0014] Preferably, in S3, the projection perspective matrix is calculated according to the four corner points of the target object coordinate system and the four corner points of the camera coordinate system, and can map any pixel coordinate to the target object coordinate system.

[0015] Preferably, in S4, the mark point of the end effector is identified by identifying the center point of the mark figure pasted thereon.

[0016] Preferably, the mark figure is rectangular, circular or cross-shaped.

[0017] Preferably, S6 specifically comprises the following steps:

[0018] S61, respectively calculating vectors formed by the start point and the end point mapped to the target object coordinate system through the two projection perspective matrices;

[0019] S62, calculating the intersection point of the two vectors;

[0020] S63, calculating the deviation of the intersection point from the center point of the target object in the target object coordinate system.

[0021] The present application has at least the following specific beneficial effects: in the case where the camera parameters are unknown, the points of the camera coordinate system are converted to the target object coordinate system by projection perspective, so as to calculate the error of the end effector of the mechanical arm from the target object. The method provides an opportunity for secondary correction for the whole system, and has high calculation accuracy, practicality and applicability without calibration. BRIEF DESCRIPTION OF DRAWINGS

[0022] Figure 1 The step flow chart of the mechanical arm motion projection correction method based on the double industrial camera of the embodiment of the present application;

[0023] Figure 2 The schematic diagram of the target object coordinate system of the mechanical arm motion projection correction method based on the double industrial camera of the embodiment of the present application;

[0024] Figure 3 The camera shooting schematic diagram of the mechanical arm motion projection correction method based on the double industrial camera of the embodiment of the present application;

[0025] Figure 4 A corresponding device structure schematic diagram of the mechanical arm motion projection correction method based on double industrial cameras according to the embodiment of the application is shown in the figure.

[0026] Figure 5 A specific flowchart of the mechanical arm motion projection correction method based on double industrial cameras according to the embodiment of the application is shown in the figure. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solutions and advantages of the application more clear, the application is further described in detail below in combination with the drawings and embodiments. In the description of the application, it should be understood that the terms "center", "longitudinal", "lateral", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the purpose of facilitating the description of the application and simplifying the description, and therefore cannot be understood as indicating or implying that the device or element indicated must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the application. In addition, the terms "first", "second" and the like are only for the purpose of description, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the technical features indicated. Therefore, the features limited by "first", "second" and the like can explicitly or implicitly include one or more features. In the description of the application, the meaning of "a plurality of" is two or more, unless otherwise specified.

[0028] It should be noted that the embodiments in the application and the features in the embodiments can be combined with each other without conflict.

[0029] The application will be further described in combination with the drawings and specific embodiments, as shown in the figure, a mechanical arm motion projection correction method based on double industrial cameras according to an embodiment of the application, the method comprises the following steps: Figure 1

[0030] S1, taking a target object as a reference, measuring the length and width of the target object;

[0031] S2, the mechanical arm moves to the vicinity of the target object actually needed to be sucked by image recognition, establishes a coordinate system of the target object, and calculates the coordinates of the four corner points and the center point of the target object according to the measured values in S1;

[0032] S3, the pixel coordinates of the four corner points of the target object are identified by two cameras respectively, and the projection perspective matrix of the two cameras is calculated together with the four corner points obtained in S2;

[0033] ​S4, two cameras are used to identify the marker points of the end effector of the robot arm, pixel coordinates are obtained and recorded as the starting point, then the robot arm is controlled to move downwards for a certain distance, and the marker points of the end effector are continuously identified as the ending point;

[0034] S5, the starting point and the ending point are mapped into the target object coordinate system through a projection perspective matrix;

[0035] S6, the deviation of the robot arm from the target object is obtained through an error calculation module.

[0036] In one embodiment, the target object numerical value measured in S1 only needs to be measured once, and subsequent measurement is not required.

[0037] In one embodiment, in S2, the right direction of the target object coordinate system is the positive direction of the X axis, and the downward direction is the positive direction of the Y axis. The unit of the coordinate system is centimeter, and the specific coordinates are consistent with the actual size of the object.

[0038] Specifically, as shown in Figure 2 , the coordinate system can take the upper left corner of the target object as the origin. In the figure, 1 represents the target object, 2 represents the center point, and the four corners on the surrounding boundary can be used as the corner points.

[0039] In one embodiment, the projection perspective matrix in S3 is calculated according to the four corner points of the target object coordinate system and the four corner points of the camera coordinate system, which can map any one pixel coordinate into the target object coordinate system.

[0040] Specifically, as shown in Figure 3 , 3 in the figure represents two cameras, 6 represents a workpiece table on which the target object is placed, 1 represents the target object, and 4 and 5 represent images of the target object taken by the two cameras. Taking the left camera as an example, a ′ 1, b1 ′ , c1 ′ , d ′ 1 corresponds to the four corner points a, b, c, and d of the target object, that is, the projection perspective matrix of the left camera is obtained through the four sets of corner points, and the right camera is the same.

[0041] Specifically, the projection perspective matrix is calculated through the following formula:

[0042]

[0043] Wherein, x', y' are the coordinates of the target object coordinate system, and x, y are the coordinates in the camera coordinate system; z' is the coordinate of the target object coordinate system, which is taken as 1. 11 , a 12 , a 13 , a 21 , a 22、a 23 、a 31 、a 32 、a 33 are all elements of the projection perspective matrix, and the matrix

[0044] Expanding the above formula we can get:

[0045]

[0046] From the formula, we can see that four sets of points are needed to solve the eight unknowns.

[0047] In one embodiment, the marker point of the end effector in S4 can be identified by identifying the center point of a marker image pasted on the end effector. The marker image can be rectangular, circular, cross-shaped, etc., and the specific selection can be based on actual conditions.

[0048] Specifically, such as Figure 4 As shown, 31 and 32 represent two cameras, 6 represents a workbench, and 1 represents a target object placed on the workbench.

[0049] Specifically, the formula for mapping pixel coordinates to the target object coordinate system is:

[0050] M1(x1,y1,1)=R*M1′(x′1,y′1,1)

[0051] M2(x2,y2,1)=R*M2′(x′2,y′2,1)

[0052] Among them, R represents the projection perspective matrix, M1 and M2 represent the coordinates of the target object coordinate system, and M1′ and M2′ represent pixel coordinates.

[0053] In one embodiment, the error calculation module in S6 includes the following steps:

[0054] S61, respectively calculating the vectors formed by the start point and the end point mapped to the target object coordinate system by the two projection perspective matrices;

[0055] S62. Calculate the intersection of two vectors;

[0056] S63: Calculate the deviation between the intersection point and the center point of the target object in the target object coordinate system.

[0057] Specifically, the formula for calculating the intersection of two vectors is as follows:

[0058]

[0059]

[0060] Among them, u, N1 ′ ,N2 ′ ,M1 ′ ,M2 ′ are four points mapped to the target object coordinate system by the projection perspective matrix.

[0061] When two vectors intersect, the following formula can be obtained:

[0062]

[0063] Cross product on both sides The formula can be obtained:

[0064]

[0065] That is, the intersection point

[0066] Specifically, the complete flow of the correction method is as shown in Figure 5 First, the target object data is measured, and then the mechanical arm is moved according to the calculation result of image recognition; then the projection perspective matrix is calculated through the corner point, and the mapping calculation deviation of the camera coordinate system and the target object coordinate system is carried out through the projection perspective matrix, if the deviation is large, then the mechanical arm is moved for correction, if the deviation is small, then it is indicated that the image recognition is accurate enough, and no correction is needed.

[0067] Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the features disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the disclosure that follow, in general, the principles of the disclosure and include other known equivalents or uses of the features shown and described. The specification and examples are to be regarded as illustrative only, and the true scope and spirit of the disclosure is indicated by the appended claims.

[0068] It should be understood that the above detailed description of the embodiments of the present application is made with reference to the accompanying drawings, but the present application is not limited to the described embodiments. For those skilled in the art, various changes, modifications, replacements and variations of the embodiments including components can be made without departing from the principles and spirits of the present application, and still fall within the protection scope of the present application. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for mechanical arm motion projection correction based on dual industrial cameras, characterized in that, The method comprises the following steps: S1, taking a target object as a reference object, measuring the length and width of the target object; S2, the robot arm moves to the vicinity of the target object to be sucked by image recognition, establishes a coordinate system of the target object, and calculates the coordinates of the four corner points and the center point of the target object according to the measured values in S1; S3, two cameras are used to identify the pixel coordinates of the four corner points of the target object, and the projection perspective matrix of the two cameras is calculated together with the four corner points obtained in S2; S4, the marker points of the end effector of the robot arm are identified by two cameras, the pixel coordinates are obtained and recorded as the starting point, then the robot arm is controlled to move downward for a certain distance, and the marker points of the end effector are identified again as the ending point; S5, the starting point and the ending point are mapped into the target object coordinate system through the projection perspective matrix; S6, the deviation of the robot arm from the target object is obtained through the error calculation module; In S2, the X-axis positive direction of the target object coordinate system is to the right, and the Y-axis positive direction is downward; the unit of the coordinate system is centimeter, and the specific coordinates are consistent with the actual size of the object; In S3, the projection perspective matrix is calculated according to the four corner points of the target object coordinate system and the four corner points of the camera coordinate system, which can map any pixel coordinate into the target object coordinate system; S6 specifically comprises the following steps: S61, the vectors formed by the starting point and the ending point mapped into the target object coordinate system through the two projection perspective matrices are calculated respectively; S62, the intersection of the two vectors is calculated; S63, the deviation of the intersection from the center point of the target object in the target object coordinate system is calculated.

2. The method of claim 1, wherein, In S1, the measurement of the target object value only needs to be measured once, and there is no need to repeat the measurement subsequently.

3. The method of claim 1, wherein, In S4, the marker points of the end effector are identified by identifying the center point of the marker diagram pasted thereon.

4. The method of claim 3, wherein, The marker diagram is a rectangle, a circle or a cross.

Images