METHOD FOR DETERMINING THE RELATIVE POSITION OF A GRIPPER

Description

FIELD OF THE INVENTION

[0001] The present invention relates generally to handling vehicles having a fork for gripping a pallet, and more particularly to a system for imaging and locating a pallet to be picked up by the vehicle. CONTEXT OF THE INVENTION

[0002] In a typical warehouse or distribution center, palletized inventory items are stored in racks or other storage structures aligned on either side of typically long, parallel aisles. To maximize available space, multiple storage structures are often stacked vertically so that inventory can be stored at elevated heights. As a result, a material handling vehicle operator retrieving and / or putting away inventory may need to look upward from a vehicle operating position to identify the height and lateral position of the forks for retrieving or putting away inventory.

[0003] When performing positioning work, the driver must operate the lifting lever while visually checking whether the forks are positioned over holes in the pallet or slightly above the surface of the platform by looking upwards (e.g., 3 to 6 meters). However, it is extremely difficult to determine whether the forks and a pallet or similar are positioned horizontally by looking upwards from below, and even a skilled person needs time for this positioning. If the forks are gradually moved towards a load or rack by, for example, operating the lifting lever, the work must be carried out with caution, thus reducing work efficiency.

[0004] Extensive training may be required to effectively make the necessary positioning adjustments. Even with sufficient capacity, proper positioning may take longer than desired for efficient use of the material handling vehicle and operator. Additionally, in warehouses or other material handling environments using fully autonomous vehicles to pick or place items, the layout and positioning of shelves and pallets tend to be strictly spaced and spaced horizontally.

[0005] Some newer forklifts have a camera attached to the truck to allow the driver to see an image of a rack or pallet in front of the forks via a display screen at the driver's seat to solve the above problem. This device makes it easier to position the forks at height.

[0006] The camera is attached to the distal end portion or side portion of the forks or to a predetermined position of the mast. State of the art

[0007] Known in the prior art is patent WO2018 / 173663 and its US national phase US2020024114 relating to a vehicle-mounted device comprising a unit for detecting a blade from a spatial recognition device, and for calculating an introduction distance which indicates the distance over which the detected blade has been introduced into an introduction target. This device is based on the acquisition of point clouds by a scanning pulsed laser.

[0008] This solution is not well-suited because it is expensive and fragile, for industrial equipment working in dusty environments and with pallets whose light reflection characteristics are not controlled. It also requires very significant computing power.

[0009] Patent application WO2018031660A1 describes another solution for locating pallets using images based on point cloud data. In one embodiment, a method for locating pallets includes acquiring, by a 3D camera, a first image of point cloud data, the first image being representative of the pallet. The method also includes generating a second image by truncating the point cloud data of the first image, and orthogonally projecting the remaining point cloud data of the first image. The method further includes generating a third image by creating a binary image from the second image, and generating a cross-correlation image by cross-correlating the third image with a model of a pallet pocket. The method also includes determining a rotation angle (R) of the pallet by analyzing the cross-correlation image.

[0010] It is also a solution implementing a lidar that is poorly suited for use on industrial equipment. Disadvantages of the prior art

[0011] Prior art solutions are based on the acquisition of point clouds by a scanning pulsed laser. Such equipment is expensive and has insufficient acquisition speed for the intended applications. Lidars are also sensitive to surrounding dust and the quality of the target reflection, which is not possible in the application considered.

[0012] Furthermore, to obtain the accuracy of the calculated position is highly dependent on the angular position acquisition relative to the fork. Indeed, the accuracy of the relative position of the fork to the pallet must be of the order of a few millimeters. For a typical distance of three meters, an angular misalignment of 2° of the optical axis results in a calculated position error of 100 millimeters. Solution provided by the invention

[0013] In order to overcome this drawback, the invention proposes, in its most general sense, a method for determining the relative position of a gripper comprising elongated male members relative to a receiver comprising female housings complementary to said elongated members, the method being defined by claim 1.

[0014] The fork reference frame is not necessarily confused with the camera reference frame (play, adjustment, alignment, etc.)

[0015] According to a first variant, said reference frame Ref p is determined with respect to a reference point 0 p,ref on a digital model of the gripper by a recognition and superposition processing of at least one characteristic zone (edge, rectangle).

[0016] According to a second variant, said reference frame Ref p is determined with respect to a reference point 0 p,ref on a digital model of the gripper by a recognition and superposition processing of a profile or a characteristic shape of the reference image and the real image.

[0017] Advantageously, said reference frame Ref p is determined with respect to a reference point 0 p,ref on a digital model of the gripper by a homothetic recognition and superposition processing of a profile or a characteristic shape of the reference image and the real image.

[0018] According to a first variant, the reference image and the real image are two-dimensional images.

[0019] According to a second variant, the reference image and the real image are three-dimensional images.

[0020] The invention also relates to the application of the method according to the invention for determining the position of a fork of a handling machine relative to a pallet.

[0021] The invention also relates to a handling machine comprising a fork and a camera for acquiring an image of the gripping area of ​​said fork, as well as a computer for carrying out processing in accordance with the invention.

[0022] Preferably, said camera is movable in height and / or laterally. Advantageously, the fork is movable laterally. Detailed description of a non-limiting example of the invention

[0023] The present invention will be better understood upon reading the detailed description of a non-limiting example of the invention which follows, referring to the appended drawings where: [ Fig. 1 ] there figure 1 represents a front view of a handling machine according to the invention [ Fig.2 ] there figure 2 represents an example of an image acquired by the camera [ Fig.3 ] there figure 3 represents an example image of a palette extracted from an image acquired by the camera [ Fig.4 ] there figure 4represents an example of angle and barycenter recognition on a pallet image extracted from an image acquired by the camera General description of the handling equipment

[0024] There figure 1 represents a front view of a handling machine according to the invention, the mechanical characteristics of which are usual, with two forks (1, 2) supported by a chassis (3) movable vertically relative to a jib (4). The chassis (3) is extended by a mesh frame (5), protecting the driver of the machine and supporting at its upper part a 360° camera (20) or a lidar for positioning the machine.

[0025] The chassis (5) supports at its lower part a camera (10) arranged in the proximal part (i.e. distant from the end of the forks (1, 2)) positioned so that its field of view includes a front part of one of the forks and the area in which the front part of the pallet is located at the time of engagement of the fork (1, 2).

[0026] This camera (10) is fixed on a vertically movable plate (11) to allow its positioning below the forks (1, 2) when the forks are in the raised position (the distance between the optical axis of the camera (10) and the lower surface of the forks (1, 2) is then approximately 300 mm), and its retraction between the forks (1, 2) when the forks are close to the ground. Image acquired by the camera (10)

[0027] There figure 2represents an example of an image acquired by the camera (10). We can distinguish the image (31, 32) of the end of the forks respectively (1, 2), and the outline of the front face of the pallet, in particular the lateral (33, 34) and middle (35) uprights, the upper face (38) and the front openings of the two housings (36, 37) in which the forks (1, 2) will have to engage. Processing of the image acquired by the camera (10)

[0028] The aim of the processing is to characterize the three-dimensional coordinates of characteristic points (for example the barycenters of the front openings) of the pallet and the forks, from a single image. These characteristic points are thus determined in the camera's frame of reference, regardless of the camera's position relative to the machine.

[0029] This information then makes it possible to determine the differences between the coordinates of the characteristic points of the pallet on the one hand, and the characteristic points of the fork on the other hand, then to calculate the trajectory of the forks to minimize these differences.

[0030] To do this, we determine from the image acquired by the camera (10) the coordinates and orientations in 6° of freedom of the fork in the camera frame of reference and the coordinates and orientations in 6° of freedom of the pallet in the camera frame of reference.

[0031] From the image whose figure 2 represents an example, we filter this image with a digital model of the front face of the pallet to extract the image of the pallet alone ( figure 3 ).

[0032] The next step is to carry out angle recognition processing (41 to 48) and barycenter (50, 51) of the areas framed by angles (41 to 44) and (45 to 48).

Claims

1. Method for determining the relative position of a gripper, which comprises elongate male elements (1, 2), with respect to a receiver, which comprises female housings which are complementary to the elongate elements (1, 2), the method including the following steps: - repeatedly acquiring an image of a physical environment in the field of a camera (10) which is positioned so that the image comprises the view of the end of respective forks (1, 2), the contour of the front face of the pallet, in particular the lateral uprights (33, 34) and central upright (35), the top face (38) and the front openings of the two housings (36, 37) in which the forks (1, 2) are to engage, and - converting, for each pixel of the image, the two-dimensional location of the pixel in the plane of the image and the depth value of the image into a three-dimensional point which corresponds to a pixel being defined by three coordinate components (x, y, z), the method further comprises: - a step of extracting the image o from a set of pixels corresponding to the gripper o from a set of pixels corresponding to the receiver the method being characterized in that it further includes: - a step of locating a reference frame Refr of the receiver, consisting of a Cartesian coordinate trihedron which has an origin 0r (xr, yr, zr, Ar, Br, Cr) in the reference frame of the camera and of which the origin corresponds to the focal point of the camera and the reference orientation corresponds to the optical axis, the reference frame Refr being determined with respect to a reference point 0r,ref on a digital model of the receiver - a step of locating a reference frame Refp of the gripper, consisting of a Cartesian coordinate trihedron which has an origin 0p (xp, yp, zp, Ap, Bp, Cp) in the reference frame of the camera and of which the origin corresponds to the focal point of the camera and the reference orientation corresponds to the optical axis, the reference frame Refp being determined with respect to a reference point 0p,ref on a digital model of the gripper - a step of calculating the relative Cartesian distance and orientation between the reference frame Refr of the receiver and said reference frame Refp of the gripper.

2. Method for determining the relative position according to claim 1, characterized in that the reference frame Refp is determined with respect to a reference point Op,ref on a digital model of the gripper by means of a process of recognition and superposition of at least one characteristic region (edge, rectangle).

3. Method for determining the relative position according to claim 1, characterized in that the reference frame Refp is determined with respect to a reference point Op,ref on a digital model of the gripper by means of a process of recognition and superposition of a profile or a characteristic shape of a reference image and the actual image.

4. Method for determining the relative position according to the preceding claim, characterized in that superposition consists of homothetic superposition of a profile or a characteristic shape of the reference image and the real image.

5. Method for determining the relative position according to claims 3 to 4, characterized in that the reference image and the actual image are two-dimensional images.

6. Method for determining the relative position according to claims 3 to 4, characterized in that the reference image and the actual image are three-dimensional images.

7. Application of the method according to any of the preceding claims for determining the position of a fork of a handling device relative to a pallet.

8. Handling device comprising a fork, a camera for acquiring an image of the gripping region of the fork, and a computer for carrying out a process in accordance with claim 1.

9. Handling device according to claim 8, characterized in that the camera is movable in height.

10. Handling device according to claim 8, characterized in that the fork is movable laterally.