METHOD FOR DETERMINING DISTANCE VALUES TO AN INDUSTRIAL TRUCK AND SUCH A

Description

[0001] The present invention relates to a method for determining distance values ​​to an industrial truck. The invention also relates to an industrial truck capable of determining distance values ​​relative to the industrial truck using a 2D camera.

[0002] The invention serves to generate three-dimensional image information of the surroundings of the industrial truck. This can be manually controlled by a vehicle operator or automatically controlled vehicles. The obtained distance values ​​can also be processed fully or semi-automatically in the industrial truck and / or visually prepared and made available to the vehicle operator. An important example where distance values ​​are very helpful is the picking up and setting down of loads, as this involves the forks of the industrial truck coming close to or closely to surrounding objects, such as shelf supports, load carriers, other vehicles, or general obstacles. Distance values ​​are also very helpful for an operator during storage and retrieval processes, enabling them to precisely control the vehicle with its forks.In addition to storing and retrieving pallets, other situations are also conceivable in which a good knowledge of the distance values ​​helps to use the vehicle more effectively.

[0003] A number of camera systems for monitoring an area in front of the load handling device are known from the state of the art. For example, it is known to use fork cameras as 2D camera systems, thus giving the operator a visual impression of the situation immediately in front of the fork tip. In addition to fork cameras, it is also known to use 3D camera systems on the forks. However, these are sensitive in design and comparatively expensive to purchase, so their use is limited.

[0004] EP 3 009 789 A1 discloses a method for determining the distance of a moving object from another object. Images of the object are captured with a 2D camera located on the moving object, and the displacement of the moving object between the images is determined using a speed-determining device. Subsequently, the change in points in the image sections of the two images is determined, and the distance of the points from the moving object is calculated using the displacement of the object. The method is primarily designed for use with vehicles.

[0005] From document EP 3 453 672 A1, it is known that a 2D camera attached to an industrial truck is used to monitor the surroundings in front of the truck in order to avoid a collision between the truck and obstacles in its path. For this purpose, images are repeatedly captured with the 2D camera during travel, and prominent points on the obstacles are identified in the images, as well as a travel distance traveled between the recordings. Using the known travel distance and the changes in the prominent points in the images, a distance of the points from the industrial truck is calculated and used to avoid collisions. However, a general detection of the surroundings for orientation purposes is not the objective. EP 3 453 672 A1 discloses the preamble of claims 1 and 7.The invention is based on the object of providing a method for an industrial truck and such a method which allows a reliable determination of distance values ​​using the simplest possible means.

[0006] According to the invention, the object is achieved by a method having the features of claim 1 and by an industrial truck having the features of claim 7. Advantageous embodiments form the subject matter of the subclaims.

[0007] The inventive method with the features of claim 1 uses a 2D camera that records images in a predetermined image area relative to the industrial truck. The 2D camera records images of a predetermined image area relative to the industrial truck. The recorded images are forwarded to an evaluation unit, which evaluates one or more distance values ​​relative to the industrial truck for at least part of the image area of ​​the recorded images. The inventive method is based on a first image being recorded with the 2D camera in a first camera position. The camera then moves a distance from the first camera position to a second camera position. A second image is then recorded with the 2D camera in the second camera position. In order to be able to evaluate the two images, it is important that the distance between the first and second camera positions is recorded.In this context, "distance" means that the distance between the first and second camera positions is recorded, and the spatial orientation of the connecting line between the first and second camera positions is available. Both are encompassed by the distance between the two camera positions. Subsequently, a distance value for at least one point and / or a range of points is determined in an overlapping area of ​​the two images, taking the distance into account. The special feature of the method according to the invention is that the images from a 2D camera are evaluated in order to obtain distance values ​​in the overlapping area of ​​the image areas.The method according to the invention is based on the realization that, especially with industrial trucks, with their typical application in a warehouse environment, changes occur comparatively slowly, so that two consecutively recorded images, in which the 2D camera was moved from a first camera position to a second camera position, can be reliably evaluated for distance values. The term "2D camera" means that a 2D image is provided by the camera, even if the camera can provide other image information. The evaluated distance value also allows a position in three-dimensional space to be determined, particularly in combination with the other image information.

[0008] In the invention, the camera is mounted on a height-adjustable part of the mast. This means that the camera moves to different heights during a lifting or lowering operation, thus assuming different height positions. The second camera position, when the industrial truck is otherwise stationary, differs only in height.

[0009] The invention also allows for a lateral camera movement, for example during a sideshift for one or both forks, whereby during the lateral camera movement the camera takes pictures from different horizontal positions.

[0010] In the invention, the camera is fixed to the industrial truck and is moved to the two camera positions by the truck's travel movement. Of course, the travel movement can also be taken into account with a 2D camera mounted on the mast. In this case, the distance between the two camera positions is determined not only by the vertical movement of the 2D camera, but also by the horizontal movement of the industrial truck, which is superimposed on it.

[0011] In the method according to the invention, a disparity map is generated for the overlap area. The disparity map provides an initial impression of which areas in the image are closer to or farther from the vehicle for image regions located in both images from different camera positions. The disparity is based on the fact that image regions identified as identical have different positions in the image when viewed from two different camera positions. Thus, the difference in the positions in the images allows for a more precise distance determination.

[0012] To use matching image areas in the two images, block matching methods are preferred. There are a number of different approaches to block matching that can be used. For the highly structured environments of industrial trucks, the use of block matching methods, particularly a semi-global block matching method, has proven particularly advantageous.

[0013] In the method according to the invention, the distance between the first and second camera positions is preferably recorded by the industrial truck. With a camera mounted on the mast, this can be done, for example, by recording the lifting / lowering speed and the duration of the lifting or lowering process. Other measurements, such as those recorded by a lift height sensor, which directly measure positions on the mast, can also be used to determine the movement.

[0014] Potential sources of error can be considered when determining the distance. For example, it is known that the mast exhibits varying degrees of deflection depending on the load and the lift height. At high lift heights, this deflection also includes a significant horizontal component and can be relevant when determining the distance.

[0015] The method according to the invention can also be extended to record a plurality of images along a trajectory of the 2D camera and forward them to the evaluation unit. The trajectory does not necessarily have to be a curved curve; a vertical movement of a 2D camera on the mast also fulfills the function of a trajectory, as does a horizontal movement of the industrial truck. Preferably, the images along the trajectory are evaluated based on the distance traveled by the camera between the images. The distance between the camera positions of two images is determined by evaluating the images recorded along the trajectory. Alternatively, methods referred to as "structure-from-motion" refer to methods in which the camera positions and their distance during recordings are estimated from prominent features in the images.

[0016] The object of the invention is also achieved by an industrial truck with the features of claim 7. The industrial truck according to the invention has a 2D camera and an evaluation unit. The images from the 2D camera are applied to the evaluation unit. Furthermore, a measuring unit is provided which records a distance for a change in the camera position on or with the industrial truck and applies this to the evaluation unit. The measuring unit can be designed depending on the type of camera movement. With a 2D camera that moves with the mast, the measuring unit can measure a change in the height position on the mast. With a 2D camera fixed to the vehicle, which changes its camera position due to a movement of the industrial truck, the measuring device can record a movement of the industrial truck and thus determine the change in the camera position.It is also possible for the measuring unit to be designed to determine a trajectory from a plurality of images and, from the determined trajectory, the distance between the individual camera images. In the industrial truck according to the invention, the evaluation unit is designed to determine a distance value for at least one point in an overlapping area of ​​the two image areas from at least two adjacent images and the distance between the camera positions of the images. The overlapping area ensures that the same point or the same range of points is recorded from two different camera positions. This makes it possible to reliably determine a distance value for cameras whose imaging ratios are known.The industrial truck according to the invention uses a simple 2D camera whose imaging ratios are known to reliably provide distance values ​​from the image area of ​​the camera after the camera has moved to the second camera position.

[0017] In the inventive design of the industrial truck, the 2D camera is mounted on a height-adjustable part of the mast. By adjusting the height of the 2D camera, it is possible to capture images at different heights on the industrial truck. It is also possible to mount the 2D camera permanently on the truck, so that the two camera positions differ from each other by a horizontal travel distance.

[0018] In the invention, even in vehicles equipped with a sideshift, images can be taken from different lateral positions from the horizontal movement of one or both forks and the different camera positions can be evaluated.

[0019] In a preferred embodiment, the measuring unit is designed to determine the distance between the camera positions from a sequence of images. The distance between two camera positions is determined using a trajectory curve along which the sequence of images was taken.

[0020] The invention is explained in more detail below using two exemplary embodiments. They show: Fig. 1: an industrial truck with a fork camera picking up pallets from the rack, Fig. 2: the use of a fork camera during sideshifting to pick up a load, Fig. 3: a vehicle-mounted camera that records a distance along the side of the industrial truck, Fig. 4: the geometric principle used to determine the distance to a point P, and Fig. 5: the principle of a pinhole camera and its imaging ratios.

[0021] Figure 1shows an industrial truck 10 equipped with a mast 12. A load-bearing device 14 is height-adjustable along the mast 12. The load-bearing device 14 has a 2D fork camera 16 on its load forks, the image area 18 of which extends beyond the fork tip. In the schematic diagram, the image area 18 extends symmetrically to the fork tip, corresponding to a camera mounted laterally on the load forks. In principle, it is also possible to provide a fork camera mounted below the load forks, the image area 18 of which is limited below the load forks.

[0022] The industrial truck 10 is located in front of a rack 20 with a load 22 to be picked up, which rests on a schematically shown pallet 24 as the load carrier. By lifting the load-bearing device 14 from time t 0 to time t 1 by a distance 26, the image area 18 transitions into an image area 18'. The image areas 18 and 18' overlap in an overlap area 28. For the overlap area 28, two images of the same object from different perspectives are thus available, which allow a distance analysis. Using the distance analysis 30, for example, the distance of the camera 16 from the front of the rack is then known. The distance of the fork tip from the rack front can thus be deduced from the distance of the camera 16 from the fork tip.

[0023] Figure 2shows a further variant for a distance measurement, in which a lateral movement of the load-bearing means takes place relative to the pallet 24 and the load 22 located thereon. When measuring the distance in Figure 2 A camera 16 is mounted on the inside of the right fork tine. The camera 16 captures a first image with the fork tine 32. In the position shifted by the distance 34, the fork tine 32' has a second position for the camera 16. This creates an overlap area 36 that provides information about the distance to the rack front. Depending on the design, for example, the lateral position of the load-bearing device relative to a central block of the pallet 24 can be captured here. The fork tines can also be aligned to the lateral boundaries of the pallet 24 as a load carrier. If the geometry of the pallets is known, this allows reliable lateral alignment of the fork tines.

[0024] Figure 3shows a further embodiment in which an industrial truck 10 moves along its longitudinal direction 38. The industrial truck 10 records an image 1 with its side camera 40 in a first position. After the industrial truck has moved into its new position as an industrial truck 10', the side camera 40 records a second image. The two image areas have an overlap area 42 in which a lateral distance to the industrial truck 10 can be calculated for one or more points. The lateral distance can be used, for example, to automatically maintain a distance to a shelf or a wall during travel. The camera 40 with the distance determination can also be used in addition to other methods, such as infrared sensors or ultrasonic measurements.

[0025] For the evaluation of the data and the determination of the distance values, Figure 4. Here, L and R denote a pinhole camera in two camera positions with parallel optical axes. For the sake of simplicity, it is assumed that the first and second camera positions are shifted along the x-axis of both pinhole cameras. Furthermore, the same focal length is denoted by f. In the first image of the pinhole camera L, the point P appears shifted by x 1 relative to the optical axis of the camera L1. As explained below, this results in a ray on which the image point lies in world coordinates. For the camera position R shifted by the distance b, the point P is shifted by the distance x 2 relative to the optical axis. The disparity of the point P is usually given as the difference between the values ​​x 1 and x 2 .

[0026] If the two images from camera positions L and R are compared in their overlapping area, a disparity map can be created. In the disparity map, equal differences for the distances from the optical axis represent areas of equal distance between the points. The distance A of point P from the X-axis is thus: A = b ⋅ f x 1 − x 2 , where b denotes the distance between the camera positions L and R, f the focal length of the camera and x 1 and x 2 the distance of the point P from the optical axis of the pinhole image in the camera positions 1 and 2.

[0027] For a better understanding, Figure 5The complete relationship of the coordinates is shown. A pinhole camera has a focal point fc with three orthogonal camera axes XC , YC , ZC . It is usually assumed that the optical axis of the pinhole camera coincides with the Z-axis of the tripod. The pinhole camera records an image in which a pixel has the coordinates U and V. As in Figure 5As shown, the image from the pinhole camera can be assumed to be at a distance of the focal length f, perpendicular to the Z-axis. From this geometric consideration, a ray can now be constructed which determines the focal point of the pinhole camera using the pixel value UV. This ray then originates from the example point P with its coordinates X, Y, Z. To understand the use of the two camera positions L and R, it is important to realize that the rays intersect in a pinhole camera image in camera position L and in camera position R. The point of intersection then uniquely determines the position of point P in the coordinate system on the ray. List of reference symbols

[0028] 10Industrial truck 12Lift mast 14Load-bearing device 162D camera 18Image area 18'Image area 20Shelf 22Load 24Pallets 26Track 28Overlap area 30Distance evaluation 32Fork 32'Fork 34Track 36Overlap area 38Vehicle longitudinal direction 40Side camera 42Overlap area

Claims

1. Method for determining distance values (A) to an industrial truck (10) which is equipped with a 2D camera (16) which records images in a predetermined image area (18) relative to the industrial truck (10) and forwards these to an evaluation unit which evaluates one or more distance values (A) relative to the industrial truck for at least part of the image area of the recorded images, the method having the following steps: ∘ Recording a first image with the 2D camera (16) in a first camera position (L), ∘ Moving the 2D camera (16) by a distance (26) from the first camera position (L) to a second camera position (R), ∘ Detection of the distance (26) between the first and the second camera position (L, R), the distance being used to detect the distance between the first and the second camera position and the spatial orientation of the connecting line from the first and the second camera position being present, ∘ Recording a second image with the 2D camera (16) in the second camera position (R), characterised in that ∘ both images are analysed, wherein an overlap area (28) of the two images between the two camera positions is determined and a distance value for at least one point (P) and / or a range of points is determined in the overlap area (28), taking into account the distance (b), ∘ the 2D camera (16) is attached to a height-adjustable part of the lifting frame (12) and is moved into the two camera positions by raising or lowering the lifting frame (12), or ∘ the 2D camera (16) is attached to a horizontally adjustable part and is moved into the two camera positions by lateral displacement, or ∘ the 2D camera (40), which detects a distance along the side of the industrial truck, is fixed to the industrial truck and the 2D camera (40) is moved into the two camera positions by a travelling movement of the industrial truck.

2. Method according to claim 1, characterised in that a disparity map is generated for the overlap area.

3. Method according to one of claims 1 to 2, characterised in that the at least one point (P) for the distance value is determined by means of a block-matching method.

4. Method according to one of claims 1 to 3, characterised in that the distance (b) between the first and the second camera position is detected by the industrial truck.

5. Method according to one of claims 1 to 4, characterised in that a plurality of images is recorded along a path curve of the 2D camera and forwarded to the evaluation unit.

6. Method according to claim 5, characterised in that the images are evaluated along the path curve to determine the distance travelled by the camera between the images.

7. Industrial truck having a 2D camera and an evaluation unit to which images from the 2D camera (16) are applied, a measuring unit being provided which detects a distance for a change in the camera position on or with the industrial truck (10) and applies it to the evaluation unit, characterised in that ∘ the evaluation unit is designed to determine a distance value for at least one point in an overlapping area of the two images from at least two adjacent images and the distance (b) between the camera positions, ∘ the 2D camera is attached to a height-adjustable part of the mast (12), or ∘ a side slider is provided to which the 2D camera (16) is attached, or ∘ the 2D camera, which detects a distance along the side of the industrial truck, is attached to the industrial truck.

8. Industrial truck according to claim 7, characterised in that the measuring unit detects the distance between the camera positions by a driving movement of the vehicle and / or by a lifting movement of the lifting frame.

9. Industrial truck according to one of claims 7 to 8, characterised in that the measuring unit is designed to determine the distance between the camera positions from a sequence of images.