Robot tray grabbing control method and system based on visual calibration

Abstract

The invention discloses a robot charging tray grabbing control method and system based on visual calibration, and relates to the technical field of charging tray grabbing control, and the method comprises the following steps: obtaining an overall image of the interior of a charging tray, and marking the overall image as a real-time charging tray image; performing graying processing on the real-time charging tray image to obtain a real-time charging tray grey-scale map; obtaining a material area based on the real-time material tray grey-scale map; acquiring a first intersection point value and a second intersection point value based on the material area; obtaining a first intersection point threshold value based on the first intersection point value; obtaining a second intersection point threshold value based on the second intersection point value; obtaining a middle fitting straight line based on the first intersection point threshold value and the second intersection point threshold value; the grabbing position of the robot is adjusted based on the middle fitting straight line; the material tray grabbing control device is used for solving the problem that in an existing material tray grabbing control technology, materials in a material tray are grabbed according to the position of the material tray, and the situation of grabbing deviation or failure is likely to occur.

Description

Technical Field

[0001] This invention relates to the field of tray gripping control technology, specifically to a robot tray gripping control method and control system based on vision calibration. Background Technology

[0002] With the rapid development of the manufacturing industry towards intelligence and flexibility, industrial robots are being used more and more widely in material handling, loading and unloading, and assembly. Among them, the robot's ability to pick up materials from the tray is a basic and key task for achieving automated production.

[0003] In traditional automated production lines, materials are neatly arranged in trays, and then a robot gripper picks up a line of materials. Typically, the gripping alignment is achieved by gripping at a fixed position on the tray, so the robot completes the gripping operation along a pre-taught fixed trajectory. This method relies on the strict positioning of the materials and the tray, and the entire system lacks flexibility and fault tolerance. If the tray position is affected by conveyor line deviations, material posture deviations in the tray grids, or different material shapes, it can cause the robot to deviate or fail to grip, or even cause damage to the workpiece or equipment. Existing tray gripping control technology grips the materials in the tray by its position, which easily leads to gripping deviations or failures. Summary of the Invention

[0004] This invention aims to at least partially solve one of the technical problems in the prior art. It involves acquiring an overall image of the inside of a material tray, marking it as a real-time material tray image; performing grayscale processing on the real-time material tray image to obtain a real-time material tray grayscale map; obtaining a material region based on the real-time material tray grayscale map; obtaining a first intersection value and a second intersection value based on the material region; obtaining a first intersection threshold based on the first intersection value; obtaining a second intersection threshold based on the second intersection value; obtaining an intermediate fitted straight line based on the first and second intersection thresholds; and adjusting the robot's gripping position based on the intermediate fitted straight line. This addresses the problem in existing material tray gripping control technology where gripping is based on the position of the material tray, leading to potential gripping offsets or failures.

[0005] To achieve the above objectives, this application provides a vision-calibrated robot tray gripping control method, comprising the following steps:

[0006] Acquire an overall image of the inside of the material tray and mark it as a real-time material tray image;

[0007] The real-time material tray image is converted to grayscale to obtain a real-time material tray grayscale image.

[0008] Material areas are obtained based on real-time grayscale images of material trays;

[0009] Obtain the first and second intersection values ​​based on the material region;

[0010] The threshold value of the first intersection point is obtained based on the value of the first intersection point.

[0011] The threshold value of the second intersection point is obtained based on the value of the second intersection point.

[0012] The intermediate fitted straight line is obtained based on the first intersection threshold and the second intersection threshold;

[0013] The robot's gripping position is adjusted based on the intermediate fitted straight line.

[0014] Furthermore, obtaining an overall image of the inside of the tray includes the following sub-steps:

[0015] Establish a planar coordinate system, labeled as the real-time planar coordinate system; place the material tray in the first quadrant of the real-time planar coordinate system, and simultaneously ensure that the column in the material tray that needs to hold the material is parallel to the Y-axis of the real-time planar coordinate system.

[0016] Furthermore, obtaining the material region based on the real-time grayscale image of the material tray includes the following sub-steps:

[0017] Obtain the grayscale values ​​of pixels in the real-time grayscale image of the material tray and mark them as tray grayscale values;

[0018] Obtain the range of grayscale values ​​for the tray, divide the range of grayscale values ​​for the tray into a first number of equal ranges, and mark them as interval division ranges;

[0019] Obtain the frequency of pallet grayscale values ​​within each interval division range and mark it as the material grayscale frequency;

[0020] A histogram is drawn with the grayscale value of the tray on the X-axis, the grayscale frequency of the material on the Y-axis, and the interval division range as the histogram interval. This histogram is then labeled as the material grayscale histogram.

[0021] Furthermore, obtaining the material region based on the real-time grayscale image of the material tray also includes the following sub-steps:

[0022] In the material grayscale histogram, the range of material grayscale frequencies greater than those on the left and right sides is marked as the first target range;

[0023] The first target range with the smallest median value is identified and marked as the second target range.

[0024] The first target range with the largest median value is identified and marked as the third target range.

[0025] The interval between the second and third target ranges with the minimum grayscale frequency of materials is identified and marked as the fourth target range.

[0026] Obtain the median value of the fourth target range and mark it as the grayscale threshold;

[0027] The grayscale values ​​of the trays that are less than the grayscale threshold are marked as the grayscale values ​​of the materials.

[0028] The area composed of the grayscale values ​​of the material is marked as the material area.

[0029] Furthermore, obtaining the first intersection value and the second intersection value based on the material region includes the following sub-steps:

[0030] Obtain the material area in the column to be grabbed from the tray and mark it as the area to be grabbed;

[0031] Starting from the X-axis of the real-time planar coordinate system, draw a second number of straight lines in the positive direction of the Y-axis at intervals of the first distance, and mark them as dividing lines;

[0032] Obtain the minimum and maximum x-coordinates of each dividing line intersecting the area to be captured, and mark them as the first intersection value and the second intersection value, respectively.

[0033] Furthermore, obtaining the second intersection threshold based on the second intersection value includes the following sub-steps:

[0034] Get the range of the first intersection value of any region to be crawled;

[0035] Establish a number line using the data at the first intersection point as the number line, and label it as the first number line;

[0036] Plot all the first intersection points on the first number line to obtain coordinate points, and mark them as the first coordinate points;

[0037] Get the length from the minimum to the maximum value of the first coordinate point on the first number line, and mark it as the first overall length;

[0038] Construct a line segment of length M1 on the first number line that can be moved, and label it as the first line segment;

[0039] Get the number of the first coordinate points in the area to be crawled, and mark them as the first overall count;

[0040] The first number threshold is calculated as: N1 = b1 × [(M1 ÷ V1) × C1]; where N1 is the first number threshold, b1 is the first ratio, V1 is the first overall length, and C1 is the first overall number.

[0041] Mark the number of the first coordinate points on the first line segment as the first search count;

[0042] Mark the leftmost end of the first line segment as the first left end;

[0043] Move the first left end to coincide with the smallest first coordinate point. If the first number of searches is less than the first number threshold, move the first line segment to the right so that the first left end coincides with the next first coordinate point. Stop moving when the first number of searches is greater than or equal to the first number threshold. Obtain the value of the first left end at this time and mark it as the first intersection threshold.

[0044] Furthermore, obtaining the second intersection threshold based on the first intersection value includes the following sub-steps:

[0045] Get the range of the second intersection point values ​​for any region to be crawled;

[0046] A number line is established using the data at the second intersection point as the number line, and this number line is marked as the second number line.

[0047] Plot all the second intersection points on the second number line to obtain coordinate points, and mark them as the second coordinate points;

[0048] Obtain the length from the minimum to the maximum value of the second coordinate point on the second number line, and mark it as the second overall length;

[0049] Construct a line segment of length M2 on the second number line that can be moved, and label it as the second line segment;

[0050] Get the number of the second coordinate points in the area to be crawled, and mark them as the second overall count;

[0051] The second threshold is calculated as: N2 = b2 × [(M2 ÷ V2) × C2]; where N2 is the second threshold, b2 is the second ratio, V2 is the second overall length, and C2 is the number of second overalls;

[0052] Mark the number of second coordinate points on the second line segment as the second search count;

[0053] Mark the rightmost end of the second line segment as the second right end;

[0054] Move the second right end to coincide with the largest second coordinate point. If the number of second searches is less than the second number threshold, move the second line segment to the left so that the second right end coincides with the next second coordinate point. Stop moving when the number of second searches is greater than or equal to the second number threshold. Obtain the value of the second right end at this time and mark it as the second intersection threshold.

[0055] Furthermore, obtaining the intermediate fitted line based on the first intersection threshold and the second intersection threshold includes the following sub-steps:

[0056] Obtain the intersection point between the dividing line corresponding to the first intersection threshold and the area to be captured, and mark it as the first filtering intersection point;

[0057] Obtain the intersection point between the dividing line corresponding to the second intersection threshold and the area to be captured, and mark it as the second filtering intersection point;

[0058] Fit all the first selected intersection points with a straight line parallel to the Y-axis to obtain a straight line, and mark it as the first fitted line;

[0059] Fit all the second-selection intersections with a straight line parallel to the Y-axis to obtain a straight line, and mark it as the second fitted line;

[0060] Find the middle parallel line between the first and second fitted lines and mark it as the middle fitted line.

[0061] Furthermore, adjusting the robot's grasping position based on the intermediate fitted line includes the following steps:

[0062] Obtain the center line parallel to the Y-axis projected onto the robot's gripper in the real-time planar coordinate system, and mark it as the gripper center line;

[0063] Adjust the robot's gripper so that its center line coincides with the fitted straight line in the middle, and then use the robot's gripper to grab the material in the tray.

[0064] This application also provides a vision-calibrated robot tray gripping control system, including: an image acquisition module, a grayscale conversion module, a region acquisition module, an intersection acquisition module, a first threshold acquisition module, a second threshold acquisition module, a function acquisition module, and a gripping adjustment module;

[0065] The image acquisition module is used to acquire an overall image of the inside of the material tray, which is marked as a real-time material tray image;

[0066] The grayscale module is used to perform grayscale processing on the real-time material tray image to obtain a real-time material tray grayscale image.

[0067] The region acquisition module is used to acquire the material region based on the real-time grayscale image of the material tray.

[0068] The intersection point acquisition module is used to acquire the first intersection point value and the second intersection point value based on the material area;

[0069] The first threshold acquisition module is used to acquire the first intersection threshold based on the first intersection value;

[0070] The second threshold acquisition module is used to acquire the second intersection threshold based on the second intersection value;

[0071] The function acquisition module is used to acquire the intermediate fitted line based on the first intersection threshold and the second intersection threshold;

[0072] The gripping adjustment module is used to adjust the robot's gripping position based on the intermediate fitted straight line.

[0073] The beneficial effects of this invention are as follows: This invention acquires an overall image of the inside of the material tray and marks it as a real-time material tray image; performs grayscale processing on the real-time material tray image to obtain a real-time material tray grayscale map; obtains the material region based on the real-time material tray grayscale map; obtains a first intersection value and a second intersection value based on the material region; obtains a first intersection threshold based on the first intersection value; obtains a second intersection threshold based on the second intersection value; obtains an intermediate fitted line based on the first intersection threshold and the second intersection threshold; and adjusts the robot's gripping position based on the intermediate fitted line. The advantage is that by adjusting the robot's gripping position based on the material in each column of the material tray, the accuracy of gripping is improved, and the occurrence of material deviation during gripping is reduced.

[0074] The present invention obtains an intermediate fitted line based on a first intersection threshold and a second intersection threshold. Its advantage is that the intermediate fitted line constructed by the material is used to adjust the gripping position and improve the gripping accuracy. Attached Figure Description

[0075] Figure 1 This is a schematic diagram of the system of the present invention;

[0076] Figure 2 This is a schematic diagram of the material grayscale histogram of the present invention;

[0077] Figure 3 This is a schematic diagram of the method for dividing straight lines according to the present invention;

[0078] Figure 4 This is a schematic diagram of the intermediate fitting line of the present invention;

[0079] Figure 5 This is a flowchart of the steps of the method of the present invention. Detailed Implementation

[0080] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0081] Example 1, please refer to Figure 1 As shown, this application provides a vision-calibrated robot tray gripping control system, including: an image acquisition module, a grayscale conversion module, a region acquisition module, an intersection acquisition module, a first threshold acquisition module, a second threshold acquisition module, a function acquisition module, and a gripping adjustment module;

[0082] The image acquisition module is used to acquire an overall image of the inside of the material tray, which is marked as a real-time material tray image; here the material tray is white and the material is black, which facilitates the acquisition of the material portion in the material tray based on the image.

[0083] The image acquisition module is configured with an image acquisition strategy, which includes:

[0084] Establish a planar coordinate system, labeled as the real-time planar coordinate system; place the material tray in the first quadrant of the real-time planar coordinate system, and ensure that the column in the material tray that needs to be clamped is parallel to the Y-axis of the real-time planar coordinate system; the real-time planar coordinate system is established to facilitate the analysis of the position of the material in the material tray.

[0085] The grayscale module is used to perform grayscale processing on the real-time material tray image to obtain a real-time material tray grayscale image.

[0086] The region acquisition module is used to acquire material regions based on real-time grayscale images of material trays;

[0087] The region acquisition module is configured with region acquisition strategies, which include:

[0088] Obtain the grayscale values ​​of pixels in the real-time grayscale image of the material tray and mark them as tray grayscale values;

[0089] Obtain the range of grayscale values ​​for the pallet, and divide the range of grayscale values ​​for the pallet into a first number of equal ranges, marking them as interval division ranges; the first number of interval division ranges are used to build a material grayscale histogram; for example, the first number is 8.

[0090] Obtain the frequency of pallet grayscale values ​​within each interval division range and mark it as the material grayscale frequency;

[0091] A histogram is drawn with the grayscale value of the tray on the X-axis, the grayscale frequency of the material on the Y-axis, and the interval division range as the histogram interval. This histogram is then marked as the material grayscale histogram.

[0092] For practical applications, please refer to Figure 2 The grayscale histogram of the material is shown.

[0093] In the material grayscale histogram, the range of material grayscale frequencies greater than those on the left and right sides is marked as the first target range;

[0094] The first target range with the smallest median value is identified and marked as the second target range.

[0095] The first target range with the largest median value is identified and marked as the third target range.

[0096] The interval between the second and third target ranges with the minimum grayscale frequency of materials is identified and marked as the fourth target range.

[0097] Obtain the median value of the fourth target range and mark it as the grayscale threshold;

[0098] The grayscale value of the tray that is less than the grayscale threshold is marked as the grayscale value of the material. Since it only contains a tray and material, the tray is white and the material is black. The grayscale value of black is less than the grayscale value of white. Therefore, the grayscale value of the tray that is less than the grayscale threshold can be marked as the grayscale value of the material. If the colors of the tray and material change, the setting can be adjusted according to the situation.

[0099] The area composed of the grayscale values ​​of the material is marked as the material area;

[0100] For practical applications, please refer to Figure 2 As shown, the second target range is 31 to 63, the third target range is 191 to 223, the fourth target range is 127 to 159, the median value of the fourth target range is 143, then the grayscale threshold is 143, and the grayscale values ​​of the pallets less than 143 are marked as material grayscale values.

[0101] The intersection point acquisition module is used to obtain the first intersection point value and the second intersection point value based on the material area;

[0102] The intersection acquisition module is configured with an intersection acquisition strategy, which includes:

[0103] Obtain the material area in the column to be grabbed from the tray and mark it as the area to be grabbed;

[0104] Starting from the X-axis of the real-time planar coordinate system, draw a second number of straight lines in the positive direction of the Y-axis at a first distance, and mark them as dividing lines; the dividing lines can cover the entire area to be grasped, with a first distance of 0.8cm and a second number of 200;

[0105] For practical applications, please refer to Figure 3 As shown, the dividing lines are drawn.

[0106] Obtain the minimum and maximum x-coordinates of each dividing line intersecting the area to be captured, and mark them as the first intersection value and the second intersection value, respectively.

[0107] The first threshold acquisition module is used to acquire the first intersection threshold based on the first intersection value;

[0108] The first threshold acquisition module is configured with a first threshold acquisition strategy, which includes:

[0109] Get the range of the first intersection value for any area to be grabbed; because if the material shape is irregular and the placement is not uniform, the size of the first intersection value will be different.

[0110] A number line is created using the first intersection point value as the number line, and this number line is labeled as the first number line. The purpose of creating the first number line is to observe the distribution of the first intersection point value.

[0111] Plot all the first intersection points on the first number line to obtain coordinate points, and mark them as the first coordinate points;

[0112] Get the length from the minimum to the maximum value of the first coordinate point on the first number line, and mark it as the first overall length;

[0113] Construct a movable line segment of length M1 on the first number line, and label it as the first line segment; the first line segment is the moving search line segment on the first number line; therefore, M1 should be set to be less than the first overall length; for example, if the first overall length is 5cm, M1 is 1cm;

[0114] Get the number of the first coordinate points in the area to be crawled, and mark them as the first overall count;

[0115] The first threshold number is calculated as: N1 = b1 × [(M1 ÷ V1) × C1]; where N1 is the first threshold number, b1 is the first ratio, V1 is the first overall length, and C1 is the first overall number; where (M1 ÷ V1) × C1 represents the average number of the first coordinate points on the first line segment. The first threshold number is set to a small value, for example, b1 is 0.5, in order to filter out sparsely distributed areas of the first coordinate points.

[0116] In practical applications, for example, if the first overall number is 20, the threshold for the first number is calculated as: N1 = 0.5 × [(1 ÷ 5) × 20] = 2, then the threshold for the first number is 2.

[0117] Mark the number of the first coordinate points on the first line segment as the first search count;

[0118] Mark the leftmost end of the first line segment as the first left end;

[0119] Move the first left end to coincide with the smallest first coordinate point. If the first number of searches is less than the first number threshold, move the first line segment to the right so that the first left end coincides with the next first coordinate point. Stop moving when the first number of searches is greater than or equal to the first number threshold. Obtain the value of the first left end at this time and mark it as the first intersection threshold. Eliminate excessively small first intersection values ​​to improve the accuracy of clamping a column of materials.

[0120] For practical applications, please refer to Figure 4 As shown, for example, the threshold for the first intersection point is 51; the threshold for the first intersection point is the minimum x-coordinate of the area to be captured after excluding excessively small first intersection point values.

[0121] The second threshold acquisition module is used to acquire the second intersection threshold based on the second intersection value;

[0122] The second threshold acquisition module is configured with a second threshold acquisition strategy, which includes:

[0123] Get the range of the second intersection point values ​​for any region to be crawled;

[0124] A number line is created using the second intersection point value as the number line, and this number line is labeled as the second number line. The purpose of creating the second number line is to observe the distribution of the second intersection point value.

[0125] Plot all the second intersection points on the second number line to obtain coordinate points, and mark them as the second coordinate points;

[0126] Obtain the length from the minimum to the maximum value of the second coordinate point on the second number line, and mark it as the second overall length;

[0127] Construct a movable line segment of length M2 on the second number line, and label it as the second line segment; the second line segment is the moving search line segment on the second number line; therefore, M2 should be set smaller than the second overall length; for example, if the second overall length is 5cm, M2 is 1cm;

[0128] Get the number of the second coordinate points in the area to be crawled, and mark them as the second overall count;

[0129] The second threshold is calculated as: N2 = b2 × [(M2 ÷ V2) × C2]; where N2 is the second threshold, b2 is the second ratio, V2 is the second overall length, and C2 is the number of second overall points; where (M2 ÷ V2) × C2 represents the average number of second coordinate points on the second line segment. The second threshold is set relatively small to filter sparsely distributed areas of second coordinate points, for example, b2 is 0.5.

[0130] In practical applications, for example, if the number of the second whole is 20, the threshold for the second number is calculated as: N2 = 0.5 × [(1 ÷ 5) × 20] = 2, then the threshold for the second number is 2.

[0131] Mark the number of second coordinate points on the second line segment as the second search count;

[0132] Mark the rightmost end of the second line segment as the second right end;

[0133] Move the second right end to coincide with the largest second coordinate point. If the number of second searches is less than the second number threshold, move the second line segment to the left so that the second right end coincides with the next second coordinate point. Stop moving when the number of second searches is greater than or equal to the second number threshold. Obtain the value of the second right end at this time and mark it as the second intersection threshold.

[0134] For practical applications, please refer to Figure 4As shown, for example, the second intersection threshold is 57; the second intersection threshold is the maximum value of the x-coordinate of the area to be captured after excluding excessively large second intersection values.

[0135] The function acquisition module is used to obtain the intermediate fitted line based on the first intersection threshold and the second intersection threshold;

[0136] The function retrieval module is configured with a function retrieval strategy, which includes:

[0137] Obtain the intersection point between the dividing line corresponding to the first intersection threshold and the area to be captured, and mark it as the first filtering intersection point;

[0138] Obtain the intersection point between the dividing line corresponding to the second intersection threshold and the area to be captured, and mark it as the second filtering intersection point;

[0139] Fit all the first screening intersections with a straight line parallel to the Y-axis to obtain a straight line, which is marked as the first fitted line; the first fitted line is the boundary line of the material in this column;

[0140] Fit all the intersections of the second screening points with a straight line parallel to the Y-axis to obtain a straight line, which is marked as the second fitted line; the second fitted line is another boundary line of the material in this column;

[0141] Obtain the middle parallel line between the first and second fitted lines and mark it as the middle fitted line; the middle fitted line is the center line of the material column, which solves the problem that the material placement position point is different from the center point of the material column when the material is placed on the tray;

[0142] For practical applications, please refer to Figure 4 As shown, the intermediate fitted line was obtained.

[0143] The gripping adjustment module is used to adjust the robot's gripping position based on the intermediate fitted straight line;

[0144] The capture and adjustment module is configured with capture and adjustment strategies, which include:

[0145] Obtain the center line parallel to the Y-axis projected onto the robot's gripper in the real-time planar coordinate system, and mark it as the gripper center line;

[0146] Adjust the robot's gripper so that its center line coincides with the fitted straight line in the middle, and then use the robot's gripper to grab the material in the tray. Compared with aligning the robot's gripper with the material placement position, grabbing the material at the fitted straight line position in the middle reduces the possibility of grabbing deviation or failure.

[0147] Example 2, please refer to Figure 5 As shown, this application provides a vision-calibrated robot tray gripping control method, including the following steps:

[0148] Step S1: Obtain an overall image of the inside of the material tray and mark it as a real-time material tray image; Step S1 includes the following sub-steps:

[0149] Step S101: Establish a planar coordinate system and mark it as the real-time planar coordinate system; place the material tray in the first quadrant of the real-time planar coordinate system, and at the same time make the column of material to be clamped in the material tray parallel to the Y-axis of the real-time planar coordinate system.

[0150] Step S2: Perform grayscale processing on the real-time material tray image to obtain a real-time material tray grayscale image.

[0151] Step S3: Obtain the material area based on the real-time grayscale image of the material tray; Step S3 includes the following sub-steps:

[0152] Step S301: Obtain the grayscale value of the pixel in the real-time material tray grayscale image and mark it as the tray grayscale value;

[0153] Step S302: Obtain the range of grayscale values ​​of the tray, divide the range of grayscale values ​​of the tray into a first number of equal ranges, and mark them as interval division ranges;

[0154] Step S303: Obtain the frequency of pallet grayscale values ​​within each interval division range and mark it as the material grayscale frequency;

[0155] Step S304: Draw a histogram with the tray grayscale value as the X-axis, the material grayscale frequency as the Y-axis, and the interval division range as the histogram interval, and mark it as the material grayscale histogram.

[0156] Step S305: In the material grayscale histogram, the range of material grayscale frequencies greater than those on the left and right sides is marked as the first target range.

[0157] Step S306: Obtain the first target range with the smallest median value among the first target ranges, and mark it as the second target range;

[0158] Step S307: Obtain the first target range with the largest median value among the first target ranges, and mark it as the third target range;

[0159] The interval between the second and third target ranges with the minimum grayscale frequency of materials is identified and marked as the fourth target range.

[0160] Step S308: Obtain the median value of the fourth target range and mark it as the grayscale threshold;

[0161] Step S309: Mark the grayscale values ​​of the trays that are less than the grayscale threshold as the grayscale values ​​of the materials;

[0162] Step S310: Mark the area composed of material grayscale values ​​as the material area.

[0163] Step S4: Obtain the first intersection value and the second intersection value based on the material region; Step S4 includes the following sub-steps:

[0164] Step S401: Obtain the material area of ​​the column to be grabbed in the material tray and mark it as the area to be grabbed;

[0165] Step S402: Starting from the X-axis of the real-time planar coordinate system, draw a second number of straight lines in the positive direction of the Y-axis at intervals of the first distance, and mark them as dividing lines;

[0166] Step S403: Obtain the minimum and maximum x-coordinates of the intersections of each dividing line with the area to be captured, and mark them as the first intersection value and the second intersection value, respectively.

[0167] Step S5: Obtain the first intersection threshold based on the first intersection value; Step S5 includes the following sub-steps:

[0168] Step S501: Obtain the range of the first intersection value of any region to be captured;

[0169] Step S502: Establish a number line using the data at the first intersection point as the number line, and mark it as the first number line;

[0170] Step S503: Plot all the first intersection point values ​​on the first number line to obtain coordinate points, and mark them as the first coordinate points;

[0171] Step S504: Obtain the length from the minimum to the maximum value of the first coordinate point on the first number line, and mark it as the first overall length;

[0172] Step S505: Construct a line segment of length M1 that can move on the first number line, and mark it as the first line segment;

[0173] Step S506: Obtain the number of the first coordinate points of the area to be captured, and mark them as the first overall count;

[0174] Step S507, calculate the first number threshold as: N1=b1×[(M1÷V1)×C1]; where N1 is the first number threshold, b1 is the first ratio, V1 is the first overall length, and C1 is the first overall number;

[0175] Step S508: Mark the number of first coordinate points on the first line segment as the first search count;

[0176] Step S509: Mark the leftmost end of the first line segment as the first left end;

[0177] Step S510: Move the first left end to coincide with the smallest first coordinate point. If the first search count is less than the first number threshold, move the first line segment to the right so that the first left end coincides with the next first coordinate point, until the first search count is greater than or equal to the first number threshold and then stop moving. Obtain the value of the first left end at this time and mark it as the first intersection threshold.

[0178] Step S6: Obtain the second intersection threshold based on the second intersection value; Step S6 includes the following sub-steps:

[0179] Step S601: Obtain the range of the second intersection point value for any region to be captured;

[0180] Step S602: Establish a number line using the data at the second intersection point as the number line, and mark it as the second number line;

[0181] Step S603: Plot all the second intersection point values ​​on the second number line to obtain coordinate points, and mark them as the second coordinate points;

[0182] Step S604: Obtain the length from the minimum to the maximum value of the second coordinate point on the second number line, and mark it as the second overall length;

[0183] Step S605: Construct a line segment of length M2 on the second number line that can be moved, and label it as the second line segment;

[0184] Step S606: Obtain the number of second coordinate points in the area to be captured, and mark them as the second overall count;

[0185] Step S607, calculate the second number threshold as: N2=b2×[(M2÷V2)×C2]; where N2 is the second number threshold, b2 is the second ratio, V2 is the second overall length, and C2 is the number of second overalls;

[0186] Step S608: Mark the number of second coordinate points on the second line segment as the second search count;

[0187] Step S609: Mark the rightmost end of the second line segment as the second right end;

[0188] Step S610: Move the second right end to coincide with the largest second coordinate point. If the number of second searches is less than the second number threshold, move the second line segment to the left so that the second right end coincides with the next second coordinate point. Stop moving when the number of second searches is greater than or equal to the second number threshold. Obtain the value of the second right end at this time and mark it as the second intersection threshold.

[0189] Step S7: Obtain the intermediate fitted line based on the first intersection threshold and the second intersection threshold; Step S7 includes the following sub-steps:

[0190] Step S701: Obtain the intersection point of the dividing line corresponding to the first intersection threshold and the area to be captured, and mark it as the first filtering intersection point;

[0191] Step S702: Obtain the intersection point of the dividing line corresponding to the second intersection threshold and the area to be captured, and mark it as the second filtering intersection point;

[0192] Step S703: Fit all the first screening intersections with a straight line parallel to the Y-axis to obtain a straight line, and mark it as the first fitted line;

[0193] Step S704: Fit all the second screening intersections with a straight line parallel to the Y-axis to obtain a straight line, and mark it as the second fitted line;

[0194] Step S705: Obtain the middle parallel line between the first fitted line and the second fitted line, and mark it as the middle fitted line.

[0195] Step S8: Adjust the robot's grasping position based on the intermediate fitted straight line; Step S8 includes the following sub-steps:

[0196] Step S801: Obtain the center line of the robot's gripper projected onto the real-time planar coordinate system, which is parallel to the Y-axis and marked as the gripper center line;

[0197] In step S802, adjust the robot's gripper so that the gripper's center line coincides with the middle fitted line, and then use the robot's gripper to grab the material in the tray.

[0198] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0199] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings or direct couplings or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

Claims

1. A robot tray grasping control method based on vision calibration, characterized in that, Includes the following steps: Acquire an overall image of the inside of the material tray and mark it as a real-time material tray image; The real-time material tray image is converted to grayscale to obtain a real-time material tray grayscale image. Material areas are obtained based on real-time grayscale images of material trays; Obtain the first and second intersection values ​​based on the material region; The threshold value of the first intersection point is obtained based on the value of the first intersection point. The threshold value of the second intersection point is obtained based on the value of the second intersection point. The intermediate fitted straight line is obtained based on the first intersection threshold and the second intersection threshold; The robot's gripping position is adjusted based on the intermediate fitted straight line.

2. The robot tray grasping control method based on vision calibration according to claim 1, characterized in that, Obtaining an overall image of the inside of the tray includes the following sub-steps: Establish a planar coordinate system, labeled as the real-time planar coordinate system; place the material tray in the first quadrant of the real-time planar coordinate system, and simultaneously ensure that the column in the material tray that needs to hold the material is parallel to the Y-axis of the real-time planar coordinate system.

3. The robot tray grasping control method based on vision calibration according to claim 2, characterized in that, Obtaining the material region based on the real-time grayscale image of the material tray includes the following sub-steps: Obtain the grayscale values ​​of pixels in the real-time grayscale image of the material tray and mark them as tray grayscale values; Obtain the range of grayscale values ​​for the tray, divide the range of grayscale values ​​for the tray into a first number of equal ranges, and mark them as interval division ranges; Obtain the frequency of pallet grayscale values ​​within each interval division range and mark it as the material grayscale frequency; A histogram is drawn with the grayscale value of the tray on the X-axis, the grayscale frequency of the material on the Y-axis, and the interval division range as the histogram interval. This histogram is then labeled as the material grayscale histogram.

4. The robot tray grasping control method based on vision calibration according to claim 3, characterized in that, Obtaining the material region based on the real-time grayscale image of the material tray also includes the following sub-steps: In the material grayscale histogram, the range of material grayscale frequencies greater than those on the left and right sides is marked as the first target range; The first target range with the smallest median value is identified and marked as the second target range. The first target range with the largest median value is identified and marked as the third target range. The interval between the second and third target ranges with the minimum grayscale frequency of materials is identified and marked as the fourth target range. Obtain the median value of the fourth target range and mark it as the grayscale threshold; The grayscale values ​​of the trays that are less than the grayscale threshold are marked as the grayscale values ​​of the materials. The area composed of the grayscale values ​​of the material is marked as the material area.

5. The robot tray grasping control method based on vision calibration according to claim 4, characterized in that, Obtaining the first and second intersection values ​​based on the material region includes the following sub-steps: Obtain the material area in the column to be grabbed from the tray and mark it as the area to be grabbed; Starting from the X-axis of the real-time planar coordinate system, draw a second number of straight lines in the positive direction of the Y-axis at intervals of the first distance, and mark them as dividing lines; Obtain the minimum and maximum x-coordinates of each dividing line intersecting the area to be captured, and mark them as the first intersection value and the second intersection value, respectively.

6. The robot tray grasping control method based on vision calibration according to claim 5, characterized in that, Obtaining the second intersection threshold based on the second intersection value includes the following sub-steps: Get the range of the first intersection value of any region to be crawled; Establish a number line using the data at the first intersection point as the number line, and label it as the first number line; Plot all the first intersection points on the first number line to obtain coordinate points, and mark them as the first coordinate points; Get the length from the minimum to the maximum value of the first coordinate point on the first number line, and mark it as the first overall length; Construct a line segment of length M1 on the first number line that can be moved, and label it as the first line segment; Get the number of the first coordinate points in the area to be crawled, and mark them as the first overall count; The first number threshold is calculated as: N1 = b1 × [(M1 ÷ V1) × C1]; where N1 is the first number threshold, b1 is the first ratio, V1 is the first overall length, and C1 is the first overall number. Mark the number of the first coordinate points on the first line segment as the first search count; Mark the leftmost end of the first line segment as the first left end; Move the first left end to coincide with the smallest first coordinate point. If the first number of searches is less than the first number threshold, move the first line segment to the right so that the first left end coincides with the next first coordinate point. Stop moving when the first number of searches is greater than or equal to the first number threshold. Obtain the value of the first left end at this time and mark it as the first intersection threshold.

7. The robot tray grasping control method based on vision calibration according to claim 6, characterized in that, Obtaining the second intersection threshold based on the first intersection value includes the following sub-steps: Get the range of the second intersection point values ​​for any region to be crawled; A number line is established using the data at the second intersection point as the number line, and this number line is marked as the second number line. Plot all the second intersection points on the second number line to obtain coordinate points, and mark them as the second coordinate points; Obtain the length from the minimum to the maximum value of the second coordinate point on the second number line, and mark it as the second overall length; Construct a line segment of length M2 on the second number line that can be moved, and label it as the second line segment; Get the number of the second coordinate points in the area to be crawled, and mark them as the second overall count; The second threshold is calculated as: N2 = b2 × [(M2 ÷ V2) × C2]; where N2 is the second threshold, b2 is the second ratio, V2 is the second overall length, and C2 is the number of second overalls; Mark the number of second coordinate points on the second line segment as the second search count; Mark the rightmost end of the second line segment as the second right end; Move the second right end to coincide with the largest second coordinate point. If the number of second searches is less than the second number threshold, move the second line segment to the left so that the second right end coincides with the next second coordinate point. Stop moving when the number of second searches is greater than or equal to the second number threshold. Obtain the value of the second right end at this time and mark it as the second intersection threshold.

8. The robot tray grasping control method based on vision calibration according to claim 7, characterized in that, Obtaining the intermediate fitted line based on the first intersection threshold and the second intersection threshold includes the following sub-steps: Obtain the intersection point between the dividing line corresponding to the first intersection threshold and the area to be captured, and mark it as the first filtering intersection point; Obtain the intersection point between the dividing line corresponding to the second intersection threshold and the area to be captured, and mark it as the second filtering intersection point; Fit all the first selected intersection points with a straight line parallel to the Y-axis to obtain a straight line, and mark it as the first fitted line; Fit all the second-selection intersections with a straight line parallel to the Y-axis to obtain a straight line, and mark it as the second fitted line; Find the middle parallel line between the first and second fitted lines and mark it as the middle fitted line.

9. The robot tray grasping control method based on vision calibration according to claim 8, characterized in that, Adjusting the robot's gripping position based on the intermediate fitted straight line includes the following steps: Obtain the center line parallel to the Y-axis projected onto the robot's gripper in the real-time planar coordinate system, and mark it as the gripper center line; Adjust the robot's gripper so that its center line coincides with the fitted straight line in the middle, and then use the robot's gripper to grab the material in the tray.

10. A vision-calibrated robot tray gripping control system, used to implement the vision-calibrated robot tray gripping control method according to any one of claims 1-9, characterized in that, include: The system includes an image acquisition module, a grayscale conversion module, a region acquisition module, an intersection point acquisition module, a first threshold acquisition module, a second threshold acquisition module, a function acquisition module, and a capture and adjustment module. The image acquisition module is used to acquire an overall image of the inside of the material tray, which is marked as a real-time material tray image; The grayscale module is used to perform grayscale processing on the real-time material tray image to obtain a real-time material tray grayscale image. The region acquisition module is used to acquire the material region based on the real-time grayscale image of the material tray. The intersection point acquisition module is used to acquire the first intersection point value and the second intersection point value based on the material area; The first threshold acquisition module is used to acquire the first intersection threshold based on the first intersection value; The second threshold acquisition module is used to acquire the second intersection threshold based on the second intersection value; The function acquisition module is used to acquire the intermediate fitted line based on the first intersection threshold and the second intersection threshold; The gripping adjustment module is used to adjust the robot's gripping position based on the intermediate fitted straight line.

Images