Bunker Reclaiming Method, Device, Reclaiming Equipment and Storage Medium

Abstract

The present invention discloses a method, device, picking equipment and storage medium for picking materials from a silo. The method for picking materials from a silo is used for the picking equipment, and the method for picking materials from a silo includes: calibrating the coordinate conversion relationship between the pixel coordinates and the world coordinates obtained by the image acquisition device, and calibrating the distance between the picking device and the image calibration point based on the coordinate conversion relationship to obtain distance information; controlling the motion device to drive the image acquisition device to move to obtain the world coordinates of at least three corner points of the tray; obtaining the acupoint position information of the material to be picked; determining the picking world coordinates at which the picking device moves to the acupoint where the material to be picked is located based on the acupoint position information, the corner point world coordinates and the distance information; controlling the motion device to drive the picking device to move to the picking world coordinates to pick up the material. By adopting the above scheme, the problems of poor accuracy, rapidity or simplicity existing in the existing picking methods are solved.

Description

Technical Field

[0001] The present invention relates to the technical field of material taking, and in particular to a material bin material taking method, device, material taking equipment and storage medium. Background Art

[0002] In the manufacturing industry, product assembly is a basic production process flow. How to quickly and accurately supply raw materials to the assembly link during the production process of products is also a very important issue. Whether the feeding can be carried out quickly will directly affect the production efficiency of products, and whether the feeding can be carried out accurately will directly affect the quality of products and the failure rate of equipment.

[0003] Currently, the main feeding methods in the feeding link include vibrating bowl feeding, tray feeding, bin feeding, etc. Among them, there are currently mainly the following 3 methods for positioning materials on a tray:

[0004] 1. Manually position a suction pen or fixture to a point at the corner of a rectangular tray, and then calculate the coordinates of each acupoint of the tray according to the number of rows and columns of the tray and the row and column spacing, so as to achieve the purpose of positioning the pick-up tray. The limitation of this method is that the row or column of the tray must be parallel to the X or Y movement direction of the suction pen or fixture, otherwise the calculated coordinates will deviate more and more as the number of rows and columns increases.

[0005] 2. Manually position a suction pen or fixture to 3 points at the corner of a rectangular tray, and then calculate the coordinates of each acupoint of the tray according to the number of rows and columns of the tray and the XY direction differences of the three points, so as to achieve the purpose of positioning the pick-up tray. The limitation of this method is that when the number of trays is large, it will take a lot of time to adjust the positions.

[0006] 3. With the aid of machine vision, each time when taking materials, the camera takes a picture of the pick-up tray, and sends the coordinates of the acupoint to be taken next time to the actuator to complete positioning and material taking. The limitation of this method is that each time when taking materials, the mechanism needs to move to the photographing position and then move to the material taking position after the camera takes a picture, wasting some time. Therefore, it is only applicable to the scenarios where material taking does not affect the overall efficiency.

[0007] The above three material taking methods have problems of poor accuracy, rapidity or simplicity, and the first two methods require manual alignment of the suction pen or fixture with the material, which requires an experienced operator to complete. Summary of the Invention

[0008] The present invention provides a material bin material taking method, device, material taking equipment and storage medium to solve the problems of poor accuracy, rapidity or simplicity existing in the existing material taking methods.

[0009] According to an aspect of the present invention, a method for retrieving materials from a silo is provided for a material retrieval device. The material retrieval device includes an image acquisition device, a motion device, a storage device, a material retrieval device, and a processing device. The image acquisition device is used to acquire images. The storage device includes a silo for placing trays. The tray has a plurality of cavities for placing materials. The material retrieval device is used to pick up the materials on the tray. The motion device is used to drive the material retrieval device and the image acquisition device to move. The processing device is electrically connected to the image acquisition device, the motion device, and the material retrieval device. The method for retrieving materials from the silo includes:

[0010] Calibrate the coordinate conversion relationship between the pixel coordinates and the world coordinates of the image acquisition device. Among them, the image acquisition device includes image calibration points;

[0011] Based on the coordinate conversion relationship, calibrate the distance between the material retrieval device and the image calibration points to obtain distance information;

[0012] Control the motion device to drive the image acquisition device to move to obtain the world coordinates of at least three corner points of the tray;

[0013] Obtain the position information of the cavity where the material to be retrieved is located;

[0014] Based on the cavity position information, the corner point world coordinates, and the distance information, determine the world coordinates for the material retrieval device to move to the cavity where the material to be retrieved is located;

[0015] Control the motion device to drive the material retrieval device to move to the world coordinates for material retrieval to pick up the material.

[0016] In an alternative embodiment of the present invention, the calibration of the coordinate conversion relationship between the pixel coordinates and the world coordinates of the image acquisition device includes:

[0017] Control the motion device to drive the image acquisition device to move to nine calibration points in turn in a nine-grid manner;

[0018] When the image acquisition device is controlled to move to a calibration point, obtain image information;

[0019] Based on the image information, use the N-point calibration algorithm to calculate the conversion relationship between the world coordinates and the pixel coordinates to obtain the coordinate conversion relationship between the pixel coordinates and the world coordinates.

[0020] In an alternative embodiment of the present invention, the material retrieval device includes a suction pen. The calibration of the distance between the material retrieval device and the image calibration points based on the coordinate conversion relationship to obtain distance information includes:

[0021] Obtain the world coordinates of the suction pen;

[0022] Control the motion device to drive the image acquisition device to move the image world coordinates until the pen is in the acquisition area of the image acquisition device;

[0023] Obtain the coaxial jig image information of the coaxial jig located at the pen acquired by the image acquisition device;

[0024] Determine the center pixel coordinates of the coaxial jig based on the coaxial jig image information;

[0025] Convert the center pixel coordinates to center world coordinates based on the coordinate conversion relationship;

[0026] Determine the distance between the pen and the image calibration point based on the pen world coordinates, the image world coordinates and the center coordinates to obtain distance information.

[0027] In an alternative embodiment of the present invention, the image acquisition device includes an upper camera and a lower camera, the number of pens is multiple, one of the multiple pens is a calibration pen, the pen world coordinates include calibration pen world coordinates, the distance information includes calibration distance information and pen distance information, and the step of determining the distance between the pen and the image calibration point based on the pen world coordinates, the image world coordinates and the center coordinates to obtain distance information includes:

[0028] Determine the distance between the calibration pen and the image calibration point based on the calibration pen world coordinates, the image world coordinates and the center coordinates to obtain calibration distance information;

[0029] Respectively obtain the pen world coordinates of the multiple pens;

[0030] Obtain the pen image information of the multiple pens acquired by the lower camera;

[0031] Respectively determine the center pixel coordinates of the multiple pens based on the multiple pen image information;

[0032] Respectively determine the pen center world coordinates based on the multiple pen center pixel coordinates through the coordinate conversion relationship;

[0033] Determine the pen distance based on the difference between the multiple pen world coordinates and the corresponding pen center world coordinates;

[0034] Determine the distance between the pens other than the calibration pen and the image calibration point based on the pen distance and the calibration distance information to obtain pen distance information.

[0035] In an alternative embodiment of the present invention, the acupoint position information of the material to be picked includes bin number information, acupoint number information, tray arrangement information, tray quantity information, tray row number information, and tray column number information; the number of trays and bins is multiple. Determining the picking world coordinate at which the picking device moves to the acupoint where the material to be picked is located based on the acupoint position information, corner point world coordinates, and the spacing information includes:

[0036] Determine the tray serial number and row-column position of the material to be picked based on the acupoint number information, the tray arrangement information, the tray quantity information, the tray row number information, and the tray column number information;

[0037] Determine the corner point coordinate information of the corresponding tray based on the tray serial number and the bin number information;

[0038] Determine the first world coordinate where the image acquisition device is located at the material to be picked based on the corner point coordinate information of at least three corner points of the corresponding tray and the row-column position of the material to be picked;

[0039] Determine the picking world coordinate at which the picking device moves to the acupoint where the material to be picked is located based on the first world coordinate and the spacing information.

[0040] In an alternative embodiment of the present invention, the number of suction pens is multiple. Before determining the picking world coordinate at which the picking device moves to the acupoint where the material to be picked is located based on the first world coordinate and the spacing information, it further includes:

[0041] Obtain the suction pen serial number;

[0042] Determine the picking suction pen based on the suction pen serial number;

[0043] Correspondingly, determining the picking world coordinate at which the picking device moves to the acupoint where the material to be picked is located based on the first world coordinate and the spacing information includes:

[0044] Determine the picking world coordinate at which the picking suction pen moves to the acupoint where the material to be picked is located based on the spacing information corresponding to the picking suction pen and the first world coordinate.

[0045] In an alternative embodiment of the present invention, the picking device further includes a pressure detection member, and the pressure detection member is arranged on the picking device;

[0046] Controlling the motion device to drive the picking device to move to the picking world coordinate to pick up the material includes:

[0047] Control the motion device to drive the picking device to move to the picking world coordinate;

[0048] Control the movement device to drive the material taking device to descend and determine in real time whether the pressure information detected by the pressure detection member is greater than a preset pressure threshold value;

[0049] When the pressure information detected by the pressure detection member is greater than the preset pressure threshold value, control the material taking device to take materials.

[0050] According to another aspect of the present invention, a material bin material taking device is provided, and the material bin material taking device includes:

[0051] A first calibration module for calibrating the coordinate conversion relationship between the pixel coordinates and the world coordinates obtained by the image acquisition device, wherein the image acquisition device includes image calibration points;

[0052] A second calibration module for calibrating the distance between the material taking device and the image calibration points based on the coordinate conversion relationship to obtain distance information;

[0053] A first control module for controlling the movement device to drive the image acquisition device to move to obtain the world coordinates of at least three corner points of the material tray;

[0054] An acquisition module for acquiring the acupoint position information of the material to be taken;

[0055] A determination module for determining the material taking world coordinates at which the material taking device moves to the acupoint where the material to be taken is located based on the acupoint position information, the corner point world coordinates, and the distance information;

[0056] A second control module for controlling the movement device to drive the material taking device to move to the material taking world coordinates to pick up materials.

[0057] According to another aspect of the present invention, a material taking device is provided, and the material taking device includes an image acquisition device, a movement device, a storage device, a material taking device, and a processing device;

[0058] The image acquisition device is used for acquiring images;

[0059] The storage device includes a material bin, and the material bin is used for placing a material tray, and the material tray has a plurality of acupoints for placing materials;

[0060] The material taking device is used for picking up the materials on the material tray, and the movement device is used for driving the material taking device and the image acquisition device to move;

[0061] The processing device is electrically connected to the image acquisition device, the movement device, and the material taking device, and the processing device is used for executing the material bin material taking method according to any embodiment of the present invention.

[0062] According to another aspect of the present invention, there is provided a computer-readable storage medium storing computer instructions for causing a processor to execute the silo material taking method according to any embodiment of the present invention.

[0063] In the technical solution of the embodiment of the present invention, the coordinate conversion relationship between the pixel coordinates and the world coordinates is obtained by calibrating the image acquisition device, wherein the image acquisition device includes image calibration points; the distance between the material taking device and the image calibration points is calibrated based on the coordinate conversion relationship to obtain distance information; the motion device is controlled to drive the image acquisition device to move to obtain the world coordinates of at least three corner points of the material tray; the acupoint position information of the material to be taken is obtained; based on the acupoint position information, the corner point world coordinates and the distance information, the material taking world coordinates where the material taking device moves to the acupoint where the material to be taken is located are determined; finally, the motion device is controlled to drive the material taking device to move to the material taking world coordinates to pick up the material. Thus, the world coordinates of each acupoint on the material tray can be quickly and accurately positioned, without being interfered by human factors, and the problems of non-horizontal or non-vertical movement directions of the material tray and the suction pen can be solved. For different batches of the same type of material tray, the difference can also be quickly calibrated and repositioned by obtaining the corner point world coordinates of three corner points again, solving the problems of poor accuracy, rapidity or simplicity existing in the existing material taking methods. The positioning problem of the material taking method for the silo material tray can be quickly and accurately completed, and repositioning can be completed within a few minutes when the point position or the hardware position changes. And when the hardware does not change, there is no need to calibrate again.

[0064] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present invention, nor is it used to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention, and those of ordinary skill in the art can also obtain other drawings without creative efforts based on these drawings.

[0066] Figure 1 is a flowchart of a silo material taking method provided in Embodiment 1 of the present invention;

[0067] Figure 2 is a flowchart of the step of calibrating the image acquisition device to obtain the coordinate conversion relationship between the pixel coordinates and the world coordinates provided in Embodiment 1 of the present invention;

[0068] Figure 3It is a flowchart of steps provided in the first embodiment of the present invention for calibrating the distance between the material taking device and the image calibration point based on the coordinate conversion relationship to obtain distance information;

[0069] Figure 4 It is a flowchart of steps provided in the first embodiment of the present invention for determining the distance between the pen suction and the image calibration point based on the pen suction world coordinates, the image world coordinates, and the center coordinates to obtain distance information;

[0070] Figure 5 It is a flowchart of steps provided in the first embodiment of the present invention for determining the material taking world coordinates at which the material taking device moves to the acupuncture point where the material to be taken is located based on the acupuncture point position information, the corner point world coordinates, and the distance information;

[0071] Figure 6 It is a flowchart of steps provided in the first embodiment of the present invention for controlling the motion device to drive the material taking device to move to the material taking world coordinates to pick up materials;

[0072] Figure 7 It is a schematic structural diagram of a bin material taking device provided in the second embodiment of the present invention;

[0073] Figure 8 It is a schematic structural diagram of a material taking device in the third embodiment of the present invention;

[0074] Figure 9 It is a schematic structural diagram of another perspective of a material taking device in the third embodiment of the present invention;

[0075] Figure 10 It is a circuit block diagram of a material taking device in the third embodiment of the present invention.

[0076] Wherein: 1. Image acquisition device; 11. Upper camera; 2. Motion device; 3. Storage device; 4. Material taking device; 41. Pen suction; 5. Processing device; 6. Pressure detection member; 71. First calibration module; 72. Second calibration module; 73. First control module; 74. Acquisition module; 75. Determination module; 76. Second control module. Detailed implementation manners

[0077] In order to enable those skilled in the art of the present technology to better understand the solution of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0078] It should be noted that the terms "first", "second", etc. in the description, claims and the above-mentioned drawings of the present invention are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present invention described here can be implemented in an order other than those illustrated or described here. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0079] Embodiment 1

[0080] Figure 1 A flowchart of a method for retrieving materials from a silo is provided for Embodiment 1 of the present invention. This embodiment is applicable to the situation where a material retrieval device retrieves materials. The material retrieval device includes an image acquisition device, a motion device, a storage device, a material retrieval device, and a processing device. The image acquisition device is used to acquire images, and the image acquisition device can be composed of a camera. The storage device includes a silo, and the silo is used to place trays. The trays have multiple cavities for placing materials. The number of silos can be multiple, and the number of trays in each silo can also be multiple, and no specific limitation is made here. The material retrieval device is used to retrieve the materials on the tray. The material retrieval device refers to a device that can retrieve the materials on the tray, such as a manipulator, a suction pen, etc. According to the different types of materials, the structure of the material retrieval device will also be different, and no specific limitation is made here. The motion device is used to drive the material retrieval device and the image acquisition device to move. In a specific embodiment, the motion device can move along the X, Y, and Z axes, that is, the motion device includes an X-axis motion component, a Y-axis motion component, and a Z-axis motion component. The processing device is electrically connected to the image acquisition device, the motion device, and the material retrieval device. This method for retrieving materials from a silo can be executed by a material retrieval device from a silo. The material retrieval device from a silo can be implemented in the form of hardware and / or software, and the material retrieval device from a silo can be configured in the processing device of the material retrieval device. As Figure 1 shown, the method for retrieving materials from a silo includes:

[0081] S110. Calibrate the image acquisition device to obtain the coordinate conversion relationship between pixel coordinates and world coordinates, where the image acquisition device includes image calibration points.

[0082] Among them, images are all composed of pixels, and pixel coordinates are the positions of pixels in the image coordinate system. The image calibration point is the origin of the image coordinate system of the image acquisition device. World coordinates are the absolute coordinates of the objective three-dimensional world, that is, they reflect the actual positions of objects in the environment. Since the pixel coordinates are obtained when the image acquisition device determines the object position by acquiring images, by calibrating the coordinate conversion relationship between the pixel coordinates and the world coordinates, it is convenient to determine the relative position between the object and the image calibration point of the image acquisition device according to the images acquired by the image acquisition device.

[0083] S120. Calibrate the distance between the material taking device and the image calibration point based on the coordinate conversion relationship to obtain distance information.

[0084] Among them, the distance information reflects the relative position differences between the material taking device and the image calibration point of the image acquisition device in each direction.

[0085] S130. Control the motion device to drive the image acquisition device to move to obtain the corner world coordinates of at least three corners of the material tray.

[0086] Among them, the material tray can be square, and the corners refer to the positions where the acupuncture points are located at the corners of the material tray. The corner world coordinates refer to the positions of the corners captured by the image acquisition device in the world coordinate system.

[0087] S140. Obtain the acupuncture point position information of the material to be taken.

[0088] Among them, the acupuncture point position information reflects the specific position of the acupuncture point where the material to be taken is located, that is, what position it is in the material tray. When the number of material bins is multiple, it also includes which bin the material tray is in, etc. When the number of material trays in each material bin is multiple, it also includes which material tray it is in this bin.

[0089] S150. Determine the material taking world coordinates at which the material taking device moves to the acupuncture point where the material to be taken is located based on the acupuncture point position information, the corner world coordinates, and the distance information.

[0090] Among them, the picking world coordinates reflect the relative position between the picking device and the acupoint where the material to be picked is located. Since the acupoints on the tray are usually arranged according to specific rules and the quantity is fixed, after knowing the corner world coordinates of at least three corner points, the acupoint world coordinates of each acupoint on the tray can be determined. By means of the acupoint position information, the acupoint where the material to be picked is located can be determined, and then the acupoint world coordinates of the acupoint where the material to be picked is located can be determined. Since the corner world coordinates are obtained by the image acquisition device, the acupoint world coordinates also reflect the relative position between the acupoint and the image calibration point of the image acquisition device, and the spacing information reflects the relative position difference between the picking device and the image calibration point of the image acquisition device in each direction. Therefore, according to the spacing information and the acupoint coordinate information of the acupoint where the material to be picked is located, the relative position between the picking device and the acupoint where the material to be picked is located, that is, the picking world coordinates, can be obtained.

[0091] S160. Control the movement device to drive the picking device to move to the picking world coordinates to pick up the material.

[0092] Among them, since the picking world coordinates reflect the relative position between the picking device and the acupoint where the material to be picked is located, the movement device can drive the picking device to move to the position where the material to be picked is located to pick up the material according to the picking world coordinates.

[0093] In the above solution, the coordinate conversion relationship between the pixel coordinates and the world coordinates is obtained by calibrating the image acquisition device, where the image acquisition device includes an image calibration point; the spacing between the picking device and the image calibration point is calibrated based on the coordinate conversion relationship to obtain spacing information; the movement device is controlled to drive the image acquisition device to move to obtain the corner world coordinates of at least three corner points of the tray; the acupoint position information of the material to be picked is obtained; the picking world coordinates for the picking device to move to the acupoint where the material to be picked is located are determined based on the acupoint position information, the corner world coordinates, and the spacing information; and finally, the movement device is controlled to drive the picking device to move to the picking world coordinates to pick up the material. Thereby, the world coordinates of each acupoint on the tray can be quickly and accurately positioned, without being interfered by human factors. The problems of non-horizontal or non-vertical movement directions of the tray and the suction pen can be solved, and the differences between different batches of the same type of tray can also be quickly calibrated and repositioned by obtaining the corner world coordinates of three corner points again, solving the problems of poor accuracy, rapidity, or simplicity existing in the existing picking methods. The positioning problem of the material picking method for the bin tray can be quickly and accurately completed. When the point position or the hardware position changes, repositioning can be completed within a few minutes. And when the hardware remains unchanged, there is no need for re-calibration.

[0094] In an alternative embodiment of the present invention, as Figure 2 shown, the obtaining the coordinate conversion relationship between the pixel coordinates and the world coordinates by calibrating the image acquisition device includes:

[0095] S111. Control the motion device to drive the image acquisition device to move to nine calibration points in sequence in a nine - grid pattern.

[0096] S112. When the image acquisition device is controlled to move to a calibration point, obtain image information.

[0097] S113. Based on the image information, use the N - point calibration algorithm to calculate the conversion relationship between the world coordinates and the pixel coordinates to obtain the coordinate conversion relationship between the pixel coordinates and the world coordinates.

[0098] Among them, the principle of the N - point calibration algorithm is to find the coordinates of N (N >= 3) identical points in two two - dimensional coordinate systems respectively, and calculate the homography matrix between the two coordinate system planes through these point coordinates. Generally, nine points are used in the project, which is commonly known as nine - point calibration. The motion device can drive the image acquisition device to move to nine calibration points in sequence in a nine - grid pattern along the X - direction and the Y - direction, and obtain image information when reaching each calibration point. Then, the N - point calibration algorithm can be used to calculate the conversion relationship between the world coordinates and the pixel coordinates to obtain the coordinate conversion relationship between the pixel coordinates and the world coordinates. In addition, the calibration point at the center of the nine - grid is the position where the image calibration point is located. In a specific embodiment, the N - point calibration algorithm can be implemented using the N - point calibration module in VisionMaster4.2.0. Through the above method, the coordinate conversion relationship between the pixel coordinates and the world coordinates can be calculated conveniently.

[0099] In an alternative embodiment of the present invention, the picking device includes a suction pen, and the suction pen refers to a component that can generate suction to adsorb materials. As Figure 3 shown, calibrating the distance between the picking device and the image calibration point based on the coordinate conversion relationship to obtain distance information includes:

[0100] S121. Obtain the world coordinates of the suction pen. Among them, the world coordinates of the suction pen refer to the position of the suction pen in the objective three - dimensional coordinates.

[0101] S122. Control the motion device to drive the image acquisition device to move the image world coordinates until the suction pen is in the acquisition area of the image acquisition device.

[0102] Among them, the acquisition area refers to the area where the image acquisition device can acquire images, and the image world coordinates refer to the coordinates of the movement of the image acquisition device, which reflects the difference between the position where the image calibration point of the image acquisition device shoots the suction pen and the original position.

[0103] S123. Obtain the coaxial jig image information of the coaxial jig located at the suction pen acquired by the image acquisition device.

[0104] Among them, the coaxial jig refers to a jig with a center for auxiliary calibration. Placing the coaxial jig at the suction pen means that the center of the coaxial jig and the center of the suction pen are located at the same position. The coaxial jig image information refers to the image including the coaxial jig.

[0105] S124. Determine the pixel coordinates of the center of the coaxial jig based on the coaxial jig image information.

[0106] Among them, the pixel coordinates of the center refer to the position of the center of the coaxial jig in the image.

[0107] S125. Convert the pixel coordinates of the center into the world coordinates of the center based on the coordinate conversion relationship.

[0108] Among them, the world coordinates of the center refer to the world coordinates generated by the pixel coordinates of the center through the coordinate conversion relationship, reflecting the relative position between the center of the coaxial jig and the image calibration point.

[0109] S126. Determine the distance between the suction pen and the image calibration point based on the world coordinates of the suction pen, the world coordinates of the image, and the coordinates of the center to obtain the distance information.

[0110] Among them, the distance information reflects the relative position between the image calibration point and the suction pen when the suction pen and the image acquisition device are in the original positions. Since the suction pen may not be within the acquisition area of the image acquisition device during installation, moving the image acquisition device driven by the motion device can make the suction pen located within the acquisition area of the image acquisition device. At this time, the image acquisition device can acquire an image including the suction pen. Furthermore, based on the world coordinates of the suction pen, the world coordinates of the image, and the coordinates of the center, the distance between the suction pen and the image calibration point can be determined to obtain the distance information.

[0111] The following uses a specific embodiment to illustrate how to obtain the distance information. In this embodiment, it is assumed that the world coordinates of the suction pen of the suction pen are (X1, Y1), the world coordinates of the image are (X2, Y2), the world coordinates of the center are (X3, Y3), and the distance information is (X 吸笔 , Y 吸笔 ).

[0112] When the installation direction of the image acquisition device is perpendicular to the motion direction of the motion device, and at this time the direction of the coordinate system inside the image acquisition device is opposite to the direction of the coordinate system of the world coordinates, then:

[0113] The distance between the suction pen and the image calibration point in the X direction is X 吸笔 = X2 + Y3 - X1.

[0114] The distance between the suction pen and the image calibration point in the Y direction is Y 吸笔 = Y2 + X3 - Y1.

[0115] When the installation direction of the image acquisition device is the same as the movement direction of the movement device, and at this time the direction of the coordinate system inside the image acquisition device is the same as the direction of the coordinate system of the world coordinate, then:

[0116] The distance between the suction pen and the image calibration point in the X direction is X 吸笔 = X2 + X3 - X1.

[0117] The distance between the suction pen and the image calibration point in the Y direction is Y 吸笔 = Y2 + Y3 - Y1.

[0118] In summary, according to the different installation directions of the image acquisition device, the distance information is determined in different ways based on the suction pen world coordinate, the image world coordinate and the center coordinate. Here, the installation direction of the image acquisition device is not specifically limited.

[0119] On the basis of the above embodiments, the image acquisition device includes an upper camera and a lower camera, the number of the suction pens is multiple, one of the multiple suction pens is a calibration suction pen, and the suction pen world coordinate includes the calibration suction pen world coordinate. The distance information includes calibration distance information and suction pen distance information. The calibration distance information reflects the relative position difference between the calibration suction pen and the image calibration point, and the suction pen world coordinate reflects the relative position difference between the other suction pens except the calibration suction pen and the image calibration point. Based on this, as Figure 4 shown, determining the distance between the suction pen and the image calibration point based on the suction pen world coordinate, the image world coordinate and the center coordinate to obtain distance information includes:

[0120] S1261. Determine the distance between the calibration suction pen and the image calibration point based on the calibration suction pen world coordinate, the image world coordinate and the center coordinate to obtain calibration distance information.

[0121] Among them, the calibration suction pen world coordinate is the actual position of the calibration suction pen in the objective three-dimensional world. The user can select different suction pens as the calibration suction pen according to their different needs. At this time, the image world coordinate refers to the coordinate when the upper camera moves to a position where it can capture the calibration suction pen, and the center coordinate refers to the coordinate of the center of the coaxial jig obtained by the upper camera shooting the coaxial jig when the coaxial jig is placed at the position of the calibration suction pen and according to the coordinate conversion relationship. The calibration distance information reflects the relative position difference between the calibration suction pen and the image calibration point. According to the above, when the calibration suction pen world coordinate, the image world coordinate and the center coordinate are known, the distance between the calibration suction pen and the image calibration point can be determined to obtain calibration distance information.

[0122] S1262. Obtain the suction pen world coordinates of the multiple suction pens respectively.

[0123] Among them, the pen suction world coordinates of multiple pen suckers reflect different positions where the different pen suckers actually are.

[0124] S1263. Obtain the pen suction image information of multiple pen suckers collected by the lower camera.

[0125] Among them, the pen suction image information refers to the images including the pen suckers. By separately photographing the pen suckers with the lower camera, the pen suction image information of multiple pen suckers can be obtained.

[0126] S1264. Respectively determine the central pixel coordinates of multiple pen suckers based on the multiple pen suction image information.

[0127] Among them, the central pixel coordinates of the pen sucker refer to the position where the center of the pen sucker is located in the pen suction image.

[0128] S1265. Respectively determine the central world coordinates of the pen suckers based on the multiple central pixel coordinates of the pen suckers through the coordinate conversion relationship.

[0129] Among them, the coordinate conversion relationship refers to the conversion relationship between pixel coordinates and world coordinates. Thus, according to the coordinate conversion relationship, the multiple central pixel coordinates of the pen suckers can be respectively converted into the central world coordinates of the pen suckers.

[0130] S1266. Determine the pen sucker spacing based on the difference between the multiple pen suction world coordinates and the corresponding central world coordinates of the pen suckers.

[0131] Among them, the pen sucker spacing refers to the spacing between the pen suckers other than the calibrated pen sucker and the calibrated pen sucker. Since multiple pen suckers are photographed and converted by the lower camera, the pen sucker spacing can be determined according to the difference between the multiple pen suction world coordinates and the corresponding central world coordinates of the pen suckers.

[0132] S1267. Determine the spacing between the remaining pen suckers other than the calibrated pen sucker and the image calibration point based on the pen sucker spacing and the calibrated spacing information to obtain the pen sucker spacing information.

[0133] Among them, the calibrated spacing information reflects the relative position difference between the calibrated pen sucker and the image calibration point, the pen suction world coordinates reflect the relative position difference between the remaining pen suckers other than the calibrated pen sucker and the image calibration point, and at the same time, the pen sucker spacing refers to the spacing between the pen suckers other than the calibrated pen sucker and the calibrated pen sucker. Therefore, according to the pen sucker spacing and the calibrated spacing information, the spacing between the remaining pen suckers other than the calibrated pen sucker and the image calibration point can be determined to obtain the pen sucker spacing information.

[0134] The following uses a specific embodiment to illustrate the specific method for obtaining the pen pick-up spacing information. Assume that the number of pen pick-ups is 5, and the 5 pen pick-ups are arranged in sequence numbers, divided into pen pick-up 1, pen pick-up 2, pen pick-up 3, pen pick-up 4, and pen pick-up 5. Select the pen pick-up that is farthest from the upper camera and the lower camera as pen pick-up 1, and determine pen pick-up 1 as the calibration pen pick-up.

[0135] Assume that the calibration pen pick-up world coordinates of the calibration pen pick-up are (X1, Y1), the image world coordinates are (X2, Y2), the center world coordinates of the circle are (X3, Y3), and the calibration spacing information is (X 吸笔1 , Y 吸笔1 ).

[0136] When the installation direction of the image acquisition device is perpendicular to the movement direction of the movement device, at this time, the direction of the coordinate system inside the image acquisition device is opposite to the direction of the world coordinate system, then:

[0137] The spacing between the calibration pen pick-up and the image calibration point in the X direction is X 吸笔1 = X2 + Y3 - X1.

[0138] The spacing between the calibration pen pick-up and the image calibration point in the Y direction is Y 吸笔1 = Y2 + X3 - Y1.

[0139] Then use the lower camera to take pictures of the 5 pen pick-ups respectively, and record the differences between the pen pick-up world coordinates and the corresponding pen pick-up center world coordinates of the multiple pen pick-ups as (X11, Y11), (X22, Y22), (X33, Y33), (X44, Y44), (X55, Y55) respectively. Then, the spacings between the remaining pen pick-ups except the calibration pen pick-up and the image calibration point (i.e., the pen pick-up spacing information) are respectively:

[0140] The spacing between pen pick-up 2 and the image calibration point X 吸笔2 = X11 - X22 + X 吸笔1。

[0141] The spacing between pen pick-up 2 and the image calibration point Y 吸笔2 = Y11 - Y22 + Y 吸笔1。

[0142] The spacing between pen pick-up 3 and the image calibration point X 吸笔3 = X11 - X33 + X 吸笔1。

[0143] The spacing between pen pick-up 3 and the image calibration point Y 吸笔3 = Y11 - Y33 + Y 吸笔1。

[0144] The spacing between pen pick-up 4 and the image calibration point X 吸笔4 = X11 - X44 + X 吸笔1。

[0145] Distance Y between the suction pen 4 and the image calibration point 吸笔4 = Y11 - Y44 + Y 吸笔1。

[0146] Distance X between the suction pen 5 and the image calibration point 吸笔5 = X11 - X55 + X 吸笔1。

[0147] Distance Y between the suction pen 5 and the image calibration point 吸笔5 = Y11 - Y55 + Y 吸笔1。

[0148] Through the above solution, in the case of multiple suction pens, the actual distances between multiple suction pens and the image calibration points of the image acquisition device can be calibrated, that is, the spacing information.

[0149] In an alternative embodiment of the present invention, the acupoint position information of the material to be picked includes bin number information, acupoint number information, tray arrangement information, tray quantity information, tray row number information, and tray column number information; the number of trays and bins is multiple, as Figure 5 shown, determining the picking world coordinates at which the picking device moves to the acupoint where the material to be picked is located based on the acupoint position information, corner point world coordinates, and the spacing information includes:

[0150] S151. Determine the tray serial number and row-column position of the material to be picked based on the acupoint number information, the tray arrangement information, the tray quantity information, the tray row number information, and the tray column number information.

[0151] Among them, the acupoint number information refers to the serial number of the acupoint where the material to be picked is located, the tray arrangement information refers to the arrangement rule of the trays when there are multiple trays, the tray quantity information refers to the number of trays, the tray row number information refers to the number of rows of each tray, and the tray column number information refers to the number of columns of each tray. The tray serial number refers to the serial number of the tray, that is, which tray among the multiple trays. The row-column position refers to which row and which column of the tray the material to be picked is in.

[0152] For example, when the acupoint number information is 20, the tray quantity information is 4, the tray arrangement information is a four-grid arrangement, that is, there are two trays in each row and two trays in each column, the tray row number information is 5, that is, each tray has 5 rows, and the tray column number information is 7, that is, each tray has 7 columns.

[0153] At this time, the hole number information is 20. There are two trays in each row, and each tray has 7 columns. Then 20 / 14 = 1 remainder 6. Each tray has 5 rows, 1 / 5 = 0 remainder 1. It can be determined that this acupuncture point is located in the first or second tray, and in the 1 + 1 = 2nd row. If the quotient is 1, it can be determined that this acupuncture point is located in the third or fourth tray. Each tray has 7 columns, 6 / 7 = 0 remainder 6. It can be determined that this acupuncture point is located in the first tray and in the 6th column. If the quotient is 1, it can be determined that this acupuncture point is located in the second tray.

[0154] In summary, it can be determined that the acupuncture point of the material to be taken is located in the 6th column of the 2nd row of the first tray.

[0155] S152. Determine the corner coordinate information of the corresponding tray based on the tray serial number and the bin number information.

[0156] Among them, the number of bins is multiple. The bin number information refers to the bin serial number where the tray where the material to be taken is located. The tray serial number refers to the serial number of the tray in the bin. Since the number of trays is multiple, the corner coordinate information of different trays will be different. According to the tray serial number and the bin number information, the tray where the material to be taken is located can be determined, and then the corner coordinate information of at least three corners of this tray obtained by the image acquisition device can be determined.

[0157] During use, at least three corners of each tray can be positioned by the image acquisition device. Then the total number of movement points of the image acquisition device is the number of trays * 3. Then save the corner world coordinates of at least three corners of each obtained tray, and carry the tray serial number and the bin number information when saving. Furthermore, when knowing the tray serial number and the bin number information, the corner world coordinates of at least three corners of this tray can be determined.

[0158] S153. Determine the first world coordinate of the image acquisition device at the position of the material to be taken based on the corner coordinate information of at least three corners of the corresponding tray and the row and column positions of the material to be taken.

[0159] Among them, after knowing the corner coordinate information of at least three corners of the tray, the coordinates of each acupuncture point on this tray can be determined. Furthermore, after knowing the row and column positions of the material to be taken, the coordinates of the acupuncture point where the material to be taken is located can be determined, that is, the first world coordinate.

[0160] For example, assuming that the horizontal is the row and the vertical is the column, the three corners are respectively the first acupuncture point in the first row of the tray, the last acupuncture point in the first column, and the last acupuncture point in the first row. The first acupuncture point is the first corner, and the corner coordinates are (X 盘1 , Y 盘1), the last acupoint in the first column is the second corner point, and the coordinate difference from the first corner point divided by (the number of rows of the tray - 1) is (X21, Y21). The last acupoint in the first row is the third corner point, and the coordinate difference from the first corner point divided by (the number of columns of the tray - 1) is (X31, Y31). Assuming that the material to be picked is located in the 2nd row and the 6th column at this time, then the first world coordinate (X 采集 , Y 采集 ) is:

[0161] X 采集 = X 盘1 + X31 * (column 6 - 1) + X21 * (row 2 - 1).

[0162] Y 采集 = Y 盘1 + Y31 * (column 6 - 1) + Y21 * (row 2 - 1).

[0163] In summary, based on the corner point coordinate information of at least three corner points of the corresponding tray and the row and column positions of the material to be picked, the first world coordinate of the image acquisition device at the position of the material to be picked can be conveniently determined.

[0164] S154. Determine the picking world coordinate at which the picking device moves to the acupoint where the material to be picked is located based on the first world coordinate and the spacing information.

[0165] Among them, since the first world coordinate reflects the relative position difference between the image calibration point of the image acquisition device and the acupoint where the material to be picked is located, the spacing information reflects the relative position difference between the image calibration point of the image acquisition device and the picking device, and the picking world coordinate reflects the relative position difference between the picking device and the acupoint where the material to be picked is located. Therefore, according to the first world coordinate and the spacing information, the picking world coordinate at which the picking device moves to the acupoint where the material to be picked is located can be determined.

[0166] For example, assuming that the spacing information is (X 间距 , Y 间距 ), and the first world coordinate (X 采集 , Y 采集 ), then the picking world coordinate (X, Y) is:

[0167] X = X 采集 - X 间距 ; Y = Y 采集 - Y 间距 .

[0168] Through the above solution, the image acquisition device can automatically move to capture at least three corner points of the tray, and then obtain the accurate position of the material at the acupuncture point. Calculating in this way can also solve the problem that there is an included angle between the row or column of the tray and the movement direction of the X or Y of the suction pen. Even if there is an included angle, the pick-up world coordinates can be accurately calculated in this way.

[0169] Optionally, when the image acquisition device includes an upper camera and a lower camera, since the upper camera is located above the material, the upper camera can be used here to obtain the corner point coordinate information of the tray.

[0170] Based on the above embodiments, the number of suction pens is multiple. Before determining the pick-up world coordinates at which the pick-up device moves to the acupuncture point where the material to be picked up is located based on the first world coordinates and the spacing information, it further includes:

[0171] Obtain the suction pen serial number.

[0172] Determine the pick-up suction pen based on the suction pen serial number.

[0173] Correspondingly, determining the pick-up world coordinates at which the pick-up device moves to the acupuncture point where the material to be picked up is located based on the first world coordinates and the spacing information includes: determining the pick-up world coordinates at which the pick-up suction pen moves to the acupuncture point where the material to be picked up is located based on the spacing information corresponding to the pick-up suction pen and the first world coordinates.

[0174] Among them, the suction pen serial number refers to the serial number of the suction pen used for picking up materials, and the pick-up suction pen refers to the suction pen used to pick up the material to be picked up this time. Through the suction pen serial number, it can be determined which one of the multiple suction pens is the pick-up suction pen used for picking up materials.

[0175] The spacing information reflects the relative position difference between the image calibration point of the image acquisition device and the pick-up device. When the number of suction pens is multiple, there will be multiple different spacing information. For example, when the suction pen includes a calibration suction pen, the spacing information includes calibration spacing information and suction pen spacing information. When the number of suction pens is greater than three, there are also three suction pen spacing information. The spacing information corresponding to the pick-up suction pen refers to the relative position difference between the suction pen used for picking up materials and the image calibration point of the image acquisition device.

[0176] Through the above solution, when there are multiple suction pens, it is also possible to accurately determine the pick-up world coordinates at which the suction pen used for picking up materials moves to the acupuncture point where the material to be picked up is located, so that picking up materials can be accurately and quickly performed.

[0177] In an alternative embodiment of the present invention, the pick-up device further includes a pressure detection member, and the pressure detection member is disposed on the pick-up device; as Figure 6As shown, controlling the motion device to drive the material taking device to move to the material taking world coordinate to pick up materials includes:

[0178] S161. Controlling the motion device to drive the material taking device to move to the material taking world coordinate.

[0179] S162. Controlling the motion device to drive the material taking device to descend and determining in real time whether the pressure information detected by the pressure detection component is greater than a preset pressure threshold.

[0180] S163. When the pressure information detected by the pressure detection component is greater than the preset pressure threshold, controlling the material taking device to pick up materials.

[0181] Among them, the pressure detection component refers to a component that can detect pressure. In a specific embodiment, the pressure detection component includes a pressure sensor. The preset pressure threshold refers to the value that the pressure information will be greater than when contacting the material. When the pressure information is greater than the preset pressure threshold, it means that the material taking device has contacted the material. By determining in real time whether the pressure information detected by the pressure detection component is greater than the preset pressure threshold, it is possible to judge whether the material to be picked up is contacted, so as to automatically determine the picking height. When the material taking device contacts the material to be picked up, picking is carried out. The existing material taking methods have the problem that the picking height needs to be aligned and set manually. When there is a slight height difference in the tray, it may cause the suction pen or fixture to damage the material or fail to obtain the material. The above solution can automatically judge whether the material is contacted by setting the pressure detection component, so as to automatically determine the picking height, effectively solving the problem of the suction pen or fixture damaging the material or failing to obtain the material.

[0182] Embodiment 2

[0183] Figure 7 The following is a schematic structural diagram of a bin material taking device provided in Embodiment 2 of the present invention. As Figure 7 shown, the device includes:

[0184] The first calibration module 71 is used to calibrate the coordinate conversion relationship between the pixel coordinates and the world coordinates obtained by the image acquisition device, where the image acquisition device includes image calibration points;

[0185] The second calibration module 72 is used to calibrate the distance between the material taking device and the image calibration points based on the coordinate conversion relationship to obtain distance information;

[0186] The first control module 73 is used to control the motion device to drive the image acquisition device to move to obtain the corner world coordinates of at least three corners of the tray;

[0187] The acquisition module 74 is used to acquire the acupoint position information of the material to be picked up;

[0188] A determination module 75, configured to determine the picking world coordinates at which the picking device moves to the acupuncture point where the material to be picked is located based on the acupuncture point position information, the corner point world coordinates, and the spacing information;

[0189] A second control module 76, configured to control the moving device to drive the picking device to move to the picking world coordinates to pick up the material.

[0190] Optionally, the first calibration module 71 includes a first control sub-module, a second control sub-module, and a calculation sub-module.

[0191] The first control sub-module is configured to control the moving device to drive the image acquisition device to move to nine calibration points in sequence in a nine-grid manner;

[0192] The second control sub-module is configured to control the image acquisition device to acquire image information when it moves to the calibration point;

[0193] The calculation sub-module is configured to calculate the conversion relationship between the world coordinates and the pixel coordinates based on the image information by using an N-point calibration algorithm to obtain the coordinate conversion relationship between the pixel coordinates and the world coordinates.

[0194] Optionally, the picking device includes a suction pen; the second calibration module 72 includes a first acquisition sub-module, a third control sub-module, a second acquisition sub-module, a first determination sub-module, a conversion sub-module, and a second determination sub-module.

[0195] The first acquisition sub-module is configured to acquire the suction pen world coordinates of the suction pen;

[0196] The third control sub-module is configured to control the moving device to drive the image acquisition device to move the image world coordinates until the suction pen is in the acquisition area of the image acquisition device;

[0197] The second acquisition sub-module is configured to acquire the coaxial jig image information of the coaxial jig located at the suction pen collected by the image acquisition device;

[0198] The first determination sub-module is configured to determine the center pixel coordinates of the coaxial jig based on the coaxial jig image information;

[0199] The conversion sub-module is configured to convert the center pixel coordinates into center world coordinates based on the coordinate conversion relationship;

[0200] The second determination sub-module is configured to determine the spacing between the suction pen and the image calibration point based on the suction pen world coordinates, the image world coordinates, and the center coordinates to obtain the spacing information.

[0201] Optionally, the image acquisition device includes an upper camera and a lower camera, the number of the suction pens is multiple, one of the multiple suction pens is a calibration suction pen, the suction pen world coordinates include the calibration suction pen world coordinates, the spacing information includes calibration spacing information and suction pen spacing information, and the second determination sub-module includes a first determination unit, an acquisition unit, a second determination unit, a third determination unit, a fourth determination unit, a fifth determination unit, and a sixth determination unit.

[0202] The first determination unit is configured to determine the spacing between the calibration suction pen and the image calibration point based on the calibration suction pen world coordinates, the image world coordinates, and the center coordinates to obtain calibration spacing information;

[0203] The acquisition unit is configured to respectively acquire the suction pen world coordinates of the multiple suction pens;

[0204] The second determination unit is configured to acquire the suction pen image information of the multiple suction pens collected by the lower camera;

[0205] The third determination unit is configured to respectively determine the center pixel coordinates of the multiple suction pens based on the multiple suction pen image information;

[0206] The fourth determination unit is configured to respectively determine the suction pen center world coordinates based on the multiple suction pen center pixel coordinates through the coordinate conversion relationship;

[0207] The fifth determination unit is configured to determine the suction pen spacing based on the difference between the multiple suction pen world coordinates and the corresponding suction pen center world coordinates;

[0208] The sixth determination unit is configured to determine the spacing between the remaining suction pens other than the calibration suction pen and the image calibration point based on the suction pen spacing and the calibration spacing information to obtain suction pen spacing information.

[0209] Optionally, the acupoint position information of the material to be picked includes bin number information, cavity number information, tray arrangement information, tray quantity information, tray row number information, and tray column number information; the number of the trays and the bins is multiple, and the determination module 75 includes a material determination sub-module, a first coordinate determination sub-module, a second coordinate determination sub-module, and a third coordinate determination sub-module.

[0210] The material determination sub-module is configured to determine the tray serial number and the row and column positions of the material to be picked based on the cavity number information, the tray arrangement information, the tray quantity information, the tray row number information, and the tray column number information;

[0211] The first coordinate determination sub-module is configured to determine the corner point coordinate information of the corresponding tray based on the tray serial number and the bin number information;

[0212] The second coordinate determination sub-module is configured to determine the first world coordinate of the image acquisition device at the material to be picked based on the corner coordinate information of at least three corner points of the corresponding tray and the row and column positions of the material to be picked;

[0213] The third coordinate determination sub-module is configured to determine the picking world coordinate of the picking device moving to the acupuncture point where the material to be picked is located based on the first world coordinate and the spacing information.

[0214] Optionally, the number of the suction pens is multiple, and the bin picking device further includes a serial number acquisition module 74 and a suction pen determination module 75.

[0215] The serial number acquisition module 74 is configured to acquire the suction pen serial number;

[0216] The suction pen determination module 75 is configured to determine the picking suction pen based on the suction pen serial number;

[0217] Correspondingly, the third coordinate determination sub-module is further configured to determine the picking world coordinate of the picking suction pen moving to the acupuncture point where the material to be picked is located based on the spacing information corresponding to the picking suction pen and the first world coordinate.

[0218] Optionally, the picking device further includes a pressure detection component, and the pressure detection component is arranged on the picking device; the second control module 76 includes a motion control sub-module, a descent control sub-module, and a picking control sub-module.

[0219] The motion control sub-module is configured to control the motion device to drive the picking device to move to the picking world coordinate;

[0220] The descent control sub-module is configured to control the motion device to drive the picking device to descend and to determine in real time whether the pressure information detected by the pressure detection component is greater than a preset pressure threshold;

[0221] The picking control sub-module is configured to control the picking device to pick up the material when the pressure information detected by the pressure detection component is greater than the preset pressure threshold.

[0222] The bin picking device provided by the embodiment of the present invention can execute the bin picking method provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the method.

[0223] Embodiment III

[0224] Figure 8 It is a schematic structural diagram of a picking device in Embodiment III of the present invention; Figure 9 It is a schematic structural diagram of another perspective of a picking device in Embodiment III of the present invention; as Figure 8 and Figure 9As shown, the material taking device includes an image acquisition device 1, a motion device 2, a material storage device 3, a material taking device 4, and a processing device 5; the image acquisition device 1 is used to acquire images; the image acquisition device 1 refers to a device that can acquire images, such as a camera, etc.

[0225] The material storage device 3 includes a silo, and the silo is used to place trays. The trays have multiple cavities for placing materials. The number of silos can be multiple, and the number of trays in each silo can also be multiple, which is not specifically limited here.

[0226] The material taking device 4 is used to pick up the materials on the tray. The material taking device 4 refers to a device that can pick up the materials on the tray, such as a manipulator, a suction pen 41, etc. According to the different types of materials, the structure of the material taking device 4 will also be different, which is not specifically limited here. The motion device 2 is used to drive the material taking device 4 and the image acquisition device 1 to move; in a specific embodiment, the motion device 2 moves along the X, Y, and Z axes, so the motion device 2 includes an X-axis motion component, a Y-axis motion component, and a Z-axis motion component.

[0227] As Figure 10 shown, the processing device 5 is electrically connected to the image acquisition device 1, the motion device 2, and the material taking device 4, and the processing device 5 is used to execute the material taking method of any embodiment of the present invention.

[0228] In the above solution, the processing device 5 calibrates the image acquisition device 1 to obtain the coordinate conversion relationship between the pixel coordinates and the world coordinates. Among them, the image acquisition device 1 includes image calibration points; based on the coordinate conversion relationship, the distance between the material taking device 4 and the image calibration points is calibrated to obtain distance information; the motion device 2 is controlled to drive the image acquisition device 1 to move to obtain the world coordinates of at least three corner points of the tray; the position information of the cavity where the material to be taken is located is obtained; based on the cavity position information, the corner point world coordinates, and the distance information, the material taking world coordinates where the material taking device 4 moves to the cavity where the material to be taken is located are determined; finally, the motion device 2 is controlled to drive the material taking device 4 to move to the material taking world coordinates to pick up the materials. Thus, it can quickly and accurately locate the world coordinates of each cavity on the tray, without being interfered by human factors, and can solve the problems of non-horizontal or non-vertical movement directions of the tray and the suction pen 41. For the differences between different batches of the same type of tray, it can also be quickly calibrated and repositioned by obtaining the corner point world coordinates of three corner points again, solving the problems of poor accuracy, rapidity, or simplicity in the existing material taking methods. It realizes the positioning problem of the material taking method for the tray in the silo, which can be repositioned within a few minutes when the point position or the hardware position changes. And when the hardware remains unchanged, there is no need for re-calibration.

[0229] In an alternative embodiment of the present invention, as Figure 9As shown in the figure, the material taking device 4 includes a suction pen 41, which is a component capable of generating suction to adsorb materials. The number of suction pens 41 can be one or more. By providing the suction pen 41, it is convenient to pick up materials with the suction pen 41.

[0230] In an alternative embodiment of the present invention, as Figure 8 shown in the figure, the image acquisition device 1 includes an upper camera 11 and a lower camera. The upper camera 11 refers to a camera located above the tray, and the lower camera refers to a camera located below the tray. By providing the upper camera 11 and the lower camera, it is convenient to take pictures of the tray and the suction pen 41 to obtain images of the tray and the suction pen 41 for calibration.

[0231] In an alternative embodiment of the present invention, as Figure 9 shown in the figure, the material taking device further includes a pressure detection member 6. The pressure detection member 6 is provided on the material taking device 4, and the processing device 5 is electrically connected to the pressure detection member 6. The processing device 5 is configured to control the material taking device 4 to take materials when the pressure information detected by the pressure detection member 6 is greater than a preset pressure threshold.

[0232] Among them, the pressure detection member 6 is a component capable of detecting pressure. In a specific embodiment, the pressure detection member 6 includes a pressure sensor. The preset pressure threshold refers to a value that the pressure information will be greater than when contacting the material. When the pressure information is greater than the preset pressure threshold, it indicates that the material taking device 4 has contacted the material. By controlling the material taking device 4 to take materials when the pressure information detected by the pressure detection member 6 is greater than the preset pressure threshold, it is possible to determine whether the material to be taken is contacted, thereby automatically determining the height of material taking. When the material taking device 4 contacts the material to be taken, it will take materials. Existing material taking methods have the problem that the height of material taking needs to be manually aligned and set. When there is a slight height difference in the tray, it may cause the suction pen 41 or the fixture to crush the material or fail to obtain the material. The above solution can automatically determine whether the material is contacted by providing the pressure detection member 6, thereby automatically determining the height of material taking, effectively solving the problem of the suction pen 41 or the fixture crushing the material or failing to obtain the material.

[0233] Embodiment Four

[0234] This Embodiment Four provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, it implements the steps of the material bin material taking method provided in any embodiment of the present invention. The method includes:

[0235] Calibrating the coordinate conversion relationship between the pixel coordinates and the world coordinates of the image acquisition device, where the image acquisition device includes image calibration points;

[0236] Based on the coordinate conversion relationship, calibrating the distance between the material taking device and the image calibration points to obtain distance information;

[0237] Control the motion device to drive the image acquisition device to move so as to obtain the corner world coordinates of at least three corner points of the tray;

[0238] Obtain the acupoint position information of the material to be picked;

[0239] Based on the acupoint position information, corner world coordinates and the spacing information, determine the picking world coordinates where the picking device moves to the acupoint where the material to be picked is located;

[0240] Control the motion device to drive the picking device to move to the picking world coordinates to pick up the material.

[0241] Certainly, a storage medium containing computer-executable instructions provided by an embodiment of the present invention, the computer-executable instructions are not limited to the method operations described above, and can also execute relevant operations in the bin picking method provided by any embodiment of the present invention.

[0242] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to: an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (non-exhaustive list) of the computer-readable storage medium include: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, the computer-readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, apparatus, or device.

[0243] The computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, which carries the computer-readable program code. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium, and this computer-readable medium can send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device.

[0244] The program code contained on the computer-readable medium can be transmitted by any suitable medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0245] Computer program code for performing the operations of the present invention may be written in one or more programming languages or combinations thereof, including object-oriented programming languages such as Java, Smalltalk, C++, as well as conventional procedural programming languages such as the "C" language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0246] It should be understood that the various forms of flow shown above may be used, with steps reordered, added, or deleted. For example, the steps recited in the present invention may be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved, and no limitation is made herein.

[0247] The above specific embodiments do not constitute a limitation on the protection scope of the present invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for retrieving materials from a silo, which is used for a material retrieval device. The material retrieval device includes an image acquisition device, a motion device, a storage device, a material retrieval device, and a processing device. The image acquisition device is used for acquiring images. The storage device includes a silo, and the silo is used for placing trays. The trays have multiple cavities for placing materials. The material retrieval device is used for picking up the materials on the trays. The motion device is used for driving the material retrieval device and the image acquisition device to move. The processing device is electrically connected to the image acquisition device, the motion device, and the material retrieval device, and is characterized in that, The silo reclaiming method comprises: Calibrate the image acquisition device to obtain a coordinate conversion relationship between pixel coordinates and world coordinates, wherein the image acquisition device includes image calibration points; Calibrate the distance between the material picking device and the image calibration point based on the coordinate transformation relationship to obtain distance information; Controlling the motion device to drive the image acquisition device to move to obtain the world coordinates of at least three corner points of the tray; Obtain the acupuncture point location information of the material to be taken; Determine the material taking world coordinates of the acupuncture point where the material to be taken is located when the material taking device moves based on the acupuncture point position information, the corner point world coordinates and the spacing information; Control the motion device to drive the material taking device to move to the material taking world coordinate to take the material; The calibrating the image acquisition device to obtain a coordinate conversion relationship between pixel coordinates and world coordinates includes: Controlling the motion device to drive the image acquisition device to move to nine calibration points in sequence in a nine-square grid manner; Controlling the image acquisition device to move to a calibration point to acquire image information; Calculating a conversion relationship between world coordinates and pixel coordinates using an N-point calibration algorithm based on the image information to obtain a coordinate conversion relationship between pixel coordinates and world coordinates; The material picking device includes a suction pen; and the distance between the material picking device and the image calibration point is calibrated based on the coordinate transformation relationship to obtain the distance information, including: Obtaining the world coordinates of the suction pen; Control the motion device to drive the image acquisition device to move the image world coordinate until the suction pen is in the acquisition area of the image acquisition device; Acquiring coaxial fixture image information of the coaxial fixture located at the suction pen captured by the image capture device; Determining the pixel coordinates of the center of the coaxial fixture based on the coaxial fixture image information; Converting the circle center pixel coordinates into circle center world coordinates based on the coordinate conversion relationship; The distance between the suction pen and the image calibration point is determined based on the world coordinates of the suction pen, the world coordinates of the image, and the world coordinates of the circle center to obtain distance information.

2. The method for retrieving materials from a silo according to claim 1, wherein The image acquisition device includes an upper camera and a lower camera, the number of the suction pens is multiple, one of the multiple suction pens is a calibration suction pens, the suction pen world coordinates include calibration suction pen world coordinates, the spacing information includes calibration spacing information and suction pen spacing information, and determining the spacing between the suction pen and the image calibration point based on the suction pen world coordinates, the image world coordinates, and the circle center world coordinates to obtain the spacing information includes: Determining the distance between the calibration pen and the image calibration point based on the calibration pen world coordinates, the image world coordinates, and the circle center world coordinates to obtain calibration distance information; respectively obtaining the world coordinates of the plurality of suction pens; Acquire suction pen image information of the plurality of suction pens captured by a lower camera; Determine a plurality of suction pen center pixel coordinates based on a plurality of suction pen image information respectively; Determine the world coordinates of the center of the suction pen respectively through the coordinate conversion relationship based on the plurality of pixel coordinates of the center of the suction pen; Determining a suction pen spacing based on a difference between a plurality of suction pen world coordinates and a corresponding suction pen center world coordinate; Based on the pen suction spacing and the calibration spacing information, determine the spacing between the remaining pen suckers other than the calibrated pen sucker and the image calibration points to obtain the pen suction spacing information.

3. The method for retrieving materials from a silo according to any one of claims 1 to 2, characterized in that The acupoint position information of the material to be picked includes bin number information, acupoint number information, tray arrangement information, number of trays information, number of rows of trays information, and number of columns of trays information; the number of trays and bins is multiple. Determining the picking world coordinates at which the picking device moves to the acupoint where the material to be picked is located based on the acupoint position information, corner point world coordinates, and the spacing information includes: Based on the acupoint number information, the tray arrangement information, the number of trays information, the number of rows of trays information, and the number of columns of trays information, determine the tray serial number and row-column position of the material to be picked; Based on the tray serial number and the bin number information, determine the corner point coordinate information of the corresponding tray; Based on the corner point coordinate information of at least three corner points of the corresponding tray and the row-column position of the material to be picked, determine the first world coordinates of the image acquisition device at the location of the material to be picked; Based on the first world coordinates and the spacing information, determine the picking world coordinates at which the picking device moves to the acupoint where the material to be picked is located.

4. The silo reclaiming method according to claim 3, characterized in that: The number of pen suckers is multiple. Before determining the picking world coordinates at which the picking device moves to the acupoint where the material to be picked is located based on the first world coordinates and the spacing information, it further includes: Obtain the pen sucker serial number; Based on the pen sucker serial number, determine the picking pen sucker; Correspondingly, determining the picking world coordinates at which the picking device moves to the acupoint where the material to be picked is located based on the first world coordinates and the spacing information includes: Based on the spacing information corresponding to the picking pen sucker and the first world coordinates, determine the picking world coordinates at which the picking pen sucker moves to the acupoint where the material to be picked is located.

5. The method for taking materials from a silo according to any one of claims 1 to 2, characterized in that, The picking device further includes a pressure detection component, and the pressure detection component is arranged on the picking device; Controlling the motion device to drive the picking device to move to the picking world coordinates to pick up the material includes: Controlling the motion device to drive the picking device to move to the picking world coordinates; Controlling the motion device to drive the picking device to descend and continuously determine whether the pressure information detected by the pressure detection component is greater than a preset pressure threshold; When the pressure information detected by the pressure detection component is greater than the preset pressure threshold, control the picking device to pick up the material.

6. A silo material taking device for performing the silo material taking method according to any one of claims 1-5, characterized in that, Includes: A first calibration module for calibrating the coordinate conversion relationship between the pixel coordinates and the world coordinates of the image acquisition device, where the image acquisition device includes image calibration points; A second calibration module for calibrating the spacing between the picking device and the image calibration points based on the coordinate conversion relationship to obtain the spacing information; A first control module for controlling the motion device to drive the image acquisition device to move to obtain the corner point world coordinates of at least three corner points of the tray; An acquisition module for acquiring the acupoint position information of the material to be picked; A determination module, configured to determine the material taking world coordinates of the acupuncture point where the material to be taken is located when the material taking device moves based on the acupuncture point position information, the corner point world coordinates and the spacing information; The second control module is used to control the movement device to drive the material taking device to move to the material taking world coordinate to take the material.

7. A material taking device, characterized in that, The material retrieving device includes an image acquisition device, a movement device, a material storage device, a material retrieving device and a processing device; The image acquisition device is used to acquire images; The storage device includes a silo for placing a material tray, and the material tray has a plurality of holes for placing materials; The material taking device is used to take the material from the material tray, and the motion device is used to drive the material taking device and the image acquisition device to move; The processing device is electrically connected to the image acquisition device, the motion device and the material retrieving device, and the processing device is used to execute the silo retrieving method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the silo retrieving method according to any one of claims 1 to 5 when executed.

Images