Industrial truck with a fork and a fork tine camera and method for operating such an industrial truck

Abstract

The invention relates to a forklift (2) and a method for operating a forklift (2). The forklift (2) comprises an assistance system and a fork (4) with at least two fork tines (61, 62). The assistance system comprises at least one fork tine camera (8, 8'), a monitor, and a processing unit (10). The fork tine camera (8, 8') is arranged on the fork (4) such that at least part of a tip (71, 72) of at least one of the two fork tines (61, 62) is within the field of view of the fork tine camera (8, 8'). The processing unit (10) is designed to capture at least one tip (71, 72) of the at least one fork tine (61, 62) in the image data, to calculate at least one mark (18) indicating a position of the fork (4) and to display image data captured by the fork tine camera (8, 8') superimposed with the mark (18) on the monitor.

Description

[0001] The invention relates to a method for operating an industrial truck with a fork with at least two forks, a fork camera, a monitor, and a processing unit for reading and processing the image data captured by the fork camera and for creating and transmitting processed image data displayed on the monitor. Furthermore, the invention relates to an industrial truck with an assistance system and with a fork with at least two forks, wherein the assistance system comprises at least one camera, a monitor, and a processing unit.

[0002] Industrial trucks, such as pallet trucks and forklifts, are often used to transport loads arranged on pallets. For this purpose, the truck comprises a fork with typically two forks, which are inserted or inserted into designated openings in a pallet to pick up the load. However, picking up a pallet and correctly inserting the forks into the designated load-handling areas of the pallet is a difficult, sometimes laborious, and time-consuming process, especially in poor visibility conditions or when the forks are raised very high. This reduces the truck's handling capacity. Furthermore, high demands are placed on the truck operator, particularly in terms of experience and skill.

[0003] An industrial truck and a control method in which the load-handling device of the industrial truck is automatically controlled and aligned to the load to be picked up are known, for example, from EP 2 653 429 B1. However, such a largely automated system is very complex.

[0004] Simpler systems include a video camera on the forks of an industrial truck. With such a system, the driver can be given a direct view of the pallet from the forks while the pallet is being picked up. This system is intended to make it easier for the driver to optimally position the forks in the load-carrying areas of the pallet. The system can be supplemented by a line laser, which is also mounted on the fork carriage. This projects the insertion height onto the pallet depending on the current position of the forks. However, there is a risk that such a line laser could be adjusted during operation of the industrial truck without the forklift driver immediately noticing. The next time the pallet is inserted, this could result in damage to the goods or the rack.

[0005] EP 3 021 178 A1 discloses a support system that displays images captured by a camera on a screen. This image information is supplemented by trajectories indicating the tips of the forks, which are determined based on the machine's steering angle.

[0006] WO 2010 / 054785 A1 discloses a guidance and collision avoidance device for forklifts. This system comprises various distance meters, the measured values ​​of which are graphically displayed on a positioning aid.

[0007] The object of the present invention is to provide a method for operating an industrial truck and an industrial truck with an assistance system, wherein the picking up of freight, in particular pallets, is to be reliable and simplified, but the system is also to be robust and less prone to errors.

[0008] The object is achieved by a method for operating an industrial truck with a fork having at least two fork tines, a fork tine camera, a monitor and a processing unit for reading out and processing the image data acquired by the fork tine camera and for creating and transmitting processed image data displayed on the monitor, wherein this method is further developed in that the fork tine camera is arranged on the fork in such a way that at least part of a tip of at least one of the two fork tines is located in the field of view of the fork tine camera, and wherein the processing unit carries out the following steps: detecting the at least one tip of the at least one fork tine in the acquired image data, calculating at least one marking which indicates a position of the fork and displaying the image data acquired by the fork tine camera, with the marking superimposed on it, on the monitor.

[0009] The fork camera recognizes at least one tip of at least one fork in the captured image data. The processing unit calculates markings, such as guide lines, to be displayed on the monitor based on the position of the forks. These guide lines help the forklift driver to better estimate the exact position of the fork tip. This makes handling the industrial truck much easier and speeds up load picking. It is advantageous for the processing unit to be able to redetermine the position of the forks at any time and thus also check whether they are still in the expected position. If, for example, a fork jumps out of position or the camera is displaced by an impact on the camera housing, either a warning can be displayed or the markings (e.g. the guide lines) can simply be recalculated and displayed depending on this new position.

[0010] A single camera is used for both image capture and for calculating the position of the fork tips. This makes the system less error-prone than conventional systems. Furthermore, multiple components can be combined. For example, the processing unit can be integrated either into the camera or into the monitor. Furthermore, it is no longer necessary to use an additional unit, such as a line laser. A digital 2D camera or a 3D camera can be used as the fork camera. It is also possible to combine a 2D camera with a distance measuring device, for example.

[0011] According to an advantageous embodiment, the upper edge(s) of the fork tines are provided as the marking. Alternatively or additionally, the outer edges of the fork tines and / or the center between the fork tines are indicated. It is also provided that the shape of the marking, for example in the center between the fork tines, consists of two vertical lines spaced apart by the width of a connecting block of a Euro pallet. This allows these two lines to be aligned with the center block of the Euro pallet when approaching the Euro pallet, allowing the fork tines to be inserted into the Euro pallet easily and reliably.

[0012] According to a further advantageous embodiment, it is provided that the processing unit further receives data relating to an angle of inclination of the fork and, on the basis of this data, calculates an auxiliary marking which indicates a deviation of the position of the fork from a zero-degree position of the fork, wherein image data captured by the fork tine camera and superimposed by the auxiliary marking are displayed on the monitor.

[0013] The processing unit receives data regarding the fork tilt angle, for example, from the industrial truck's operating control system. For this purpose, the processing unit is configured to retrieve the required data, for example. It is also designed to request the data in question (i.e., send a request) and receive the data shortly afterward in response to this request. The data can also be received via push notification without being requested. The zero-degree position of the fork is specifically a position in which the fork tines are aligned horizontally.

[0014] When the fork is in the zero-degree position, the auxiliary mark and the mark are aligned. With this fork alignment, the fork tines are inserted horizontally into the pallet. If the auxiliary mark and the mark diverge, the fork tips are inserted into the pallet in the area of ​​the auxiliary mark. However, the tops of the fork tines will then come into contact with the pallet, preventing the fork from being fully inserted. This may be desirable in certain circumstances. The auxiliary mark, like the mark, can be a line indicating the top edge of the fork tines, for example.

[0015] Furthermore, the method is particularly further developed in that the processing unit further receives data relating to a steering angle and, on the basis of this data, calculates a further auxiliary marking which indicates a future deviation of the position of the fork from its current position in a conveying plane during an assumed journey below the steering angle, wherein image data captured by the fork camera and superimposed by the further auxiliary marking are displayed on the monitor.

[0016] The processing unit also receives data regarding the steering angle, for example, from the industrial truck's control system. The effects of the currently set steering angle can be estimated in advance by the forklift driver. This allows the corresponding steering angle to be deliberately used or avoided altogether.

[0017] In addition to a horizontal line, the marking can include, for example, a vertical line indicating the center between the fork tine tips. The additional auxiliary marking can also be a vertical line, which can be distinguished from the vertical line of the main marking, for example, by color. The larger the selected steering angle, the greater the deviation between the vertical auxiliary marking and the vertical main marking.

[0018] According to a further embodiment, it is provided that a load to be picked up with at least two visible load-carrying areas is present in the captured image data, wherein target positions of the tips of the forks are calculated when the forks penetrate into the visible load-carrying areas and image data captured by the fork camera are displayed on the monitor in overlay with the calculated target position of the tips of the forks.

[0019] For example, schematically indicated forks can be displayed as the target positions of the fork tips. Their position is continuously updated as the fork's distance from the load decreases. This allows the forklift operator to gradually align or maintain the virtual positions of the forks with the actual positions of the forks, as observed in the live image from the fork camera, so that optimal penetration into the pallet can be achieved. This significantly simplifies operation of the industrial truck for the forklift operator.

[0020] According to a further embodiment, live image data is captured by the fork camera, the processing unit executes the aforementioned steps in real time, and an overlay of a live image and the marking calculated in real time is displayed on the monitor. This may be modified, if necessary, to calculate the fork positions and the resulting positions of the markings, in particular vertical and / or horizontal, in particular main lines and / or auxiliary lines, and remain valid or displayed as long as the fork tines are not adjusted.

[0021] The fork camera is, in particular, a digital camera. The processing unit is a computing unit, such as an embedded PC.

[0022] The object is further achieved by an industrial truck with an assistance system and with a fork with at least two forks, wherein the assistance system comprises at least one fork camera, a monitor and a processing unit, wherein the industrial truck is further developed in that the fork camera is arranged on the fork in such a way that at least part of a tip of at least one of the two forks is in the field of view of the fork camera, and wherein the processing unit is configured to capture at least one tip of the at least one fork in the image data, to calculate at least one marking that indicates a position of the fork, and to display image data captured by the fork camera, which are superimposed on the marking, on the monitor.

[0023] The same or similar advantages apply to the industrial truck as those already mentioned with regard to the method of operating the industrial truck, so repetition will be avoided.

[0024] The industrial truck is particularly further developed in that the marking indicates the top sides and / or the outer edges of the forks and / or the center between the forks.

[0025] According to a further advantageous embodiment, the industrial truck is further developed in that the processing unit is further configured to receive data relating to an angle of inclination of the fork and, on the basis of this data, to calculate an auxiliary marking which indicates a deviation of the position of the fork from a zero-degree position of the fork and to display image data captured by the fork camera, which are superimposed on the auxiliary marking, on the monitor.

[0026] Furthermore, it is particularly provided that the industrial truck is further developed in that the processing unit is further configured to receive data relating to a steering angle and, on the basis of this data, to calculate a further auxiliary marking which indicates a future deviation of the position of the fork from its current position in a conveying plane during an assumed journey at the steering angle, and to display image data captured by the fork camera, which are superimposed by the further auxiliary marking, on the monitor.

[0027] According to a further advantageous embodiment, it is provided that a load to be picked up with at least two visible load-carrying areas is present in the captured image data, wherein the processing unit is further configured to calculate target positions of the tips of the forks when inserting into the visible load-carrying areas and to display image data captured by the fork camera on the monitor in overlay with the calculated target position of the tips of the forks.

[0028] Finally, the industrial truck is advantageously further developed in that the fork camera is configured to capture live image data, and the processing unit is configured to perform the aforementioned steps in real time and display an overlay of a live image and the marking calculated in real time on the monitor. This may be modified if necessary to calculate the fork positions and the resulting positions of the markings, in particular vertical and / or horizontal, in particular main lines and / or auxiliary lines, and remain valid or displayed as long as the fork arms are not adjusted.

[0029] Further features of the invention will become apparent from the description of embodiments of the invention together with the claims and the accompanying drawings. Embodiments of the invention may incorporate individual features or a combination of several features.

[0030] The invention is described below, without limiting the general inventive concept, using exemplary embodiments with reference to the drawings, whereby express reference is made to the drawings for all details of the invention not explained in more detail in the text. They show: Fig. 1 shows an industrial truck in a schematically simplified perspective view, Fig. 2 shows a pallet for a load to be picked up in a schematically simplified perspective view, Fig. 3 shows an exemplary image displayed on the monitor in which the image captured by the fork camera is shown superimposed with a marking, wherein the main marking and the auxiliary marking differ from one another, Fig. 4 shows an exemplary image displayed on the monitor in which the image captured by the fork camera is shown superimposed with a marking, wherein the main marking and the auxiliary marking are in register, and Fig. 5 shows image data captured by the fork camera superimposed with calculated target positions of the tips of the forks, wherein the image shows a schematically represented shelf.

[0031] In the drawings, identical or similar elements and / or parts are provided with the same reference numbers, so that a repeated presentation is omitted.

[0032] Fig. 1 shows, in a schematically simplified perspective view, an industrial truck 2 with a fork 4 with at least two fork tines 61, 62 and a fork tine camera 8, 8' mounted on the fork 4. In the present exemplary embodiment, a fork tine camera 8 can be arranged either on the back of the fork or, in the form of an alternative or additional placement as a fork tine camera 8', directly on the fork 4, in this case the inside of a fork tine 61.

[0033] Furthermore, the industrial truck 2, which is, for example, a forklift truck, comprises a monitor (not shown in the figure) which is located in the field of view of a driver of the industrial truck 2. Furthermore, a processing unit 10 is included, which is indicated schematically. This serves to read out and process the image data acquired by the fork camera 8, 8' and to create and transmit processed image data displayed on the monitor. Deviating from the exemplary representation in Fig. 1The processing unit 10 can also be integrated into the fork camera 8, 8' or into the monitor. The fork camera 8, 8' is arranged on the fork 4 of the industrial truck 2 such that at least part of a tip 71, 72 of at least one of the two forks 61, 62 is in the field of view of the fork camera 8, 8'. The industrial truck 2 comprises an assistance system that includes at least the fork camera 8, 8', the monitor, and the processing unit 10.

[0034] Fig. 2shows a schematically simplified perspective view of a pallet 12, such as is used to pick up goods and is stored, for example, in a high-bay warehouse. The industrial truck 2 picks up such a pallet 12 in the load-carrying areas 16a, 16b located between adjacent pallet blocks 14a, 14b, 14c. For this purpose, the forks 61, 62 are inserted into the corresponding load-carrying areas 16a, 16b. This requires considerable skill in handling the industrial truck 2, especially in poor lighting or at great working heights. The fork camera 8, 8' provides the driver of the industrial truck 2 with an image on the monitor, which facilitates pallet insertion.

[0035] Fig. 3shows an example of an image displayed on the monitor of the industrial truck 2 in a simplified schematic representation. In the display on the monitor, the image captured by the fork camera 8, 8' is superimposed with various markings 18.

[0036] The inner sides of the fork tines 61, 62 are visible in a viewing direction towards the respective tips 71, 72 of the fork tines 61, 62. A first and a second main marking 20a, 20b and a first and a second auxiliary marking 22a, 22b are provided as markings 18. The first main marking 20a, a horizontal line shown in color, for example, indicates a position of an upper side 24 (see also Fig. 1 ) of the fork tines 61, 62. The position of this main marking 20a, which indicates a position of the fork 4, is calculated by the processing unit 10 from the image data captured by the fork tine camera 8, 8'.

[0037] It is also provided, by way of example, that the outer edges of the fork tines 61, 62 are displayed as marking 18. However, this will be discussed in connection with Fig. 5 be addressed. In Fig. 3 A second main marking 20b is visible, which indicates a center between the two fork tines 61, 62. This is, for example, a colored, illuminated vertical line. When approaching the example shown in Fig. 2 The driver of the industrial truck 2 can aim at the middle pallet block 14b with the second main marking 20b on the pallet 12 shown, so that he can insert the tips 71, 72 of the forks 61, 62 safely and reliably into the load-carrying areas 16a, 16b present on both sides of this middle pallet block 14b.

[0038] The processing unit 10 further receives data concerning an inclination angle of the fork 4 and calculates the auxiliary mark 22a on the basis of this data. Fig. 3shows a situation in which the fork 4 of the industrial truck 2 is tilted downwards from the horizontal, for example. This means that the tips 71, 72 of the fork tines 61, 62 are tilted downwards from a horizontal plane. With such a position of the fork 4, the upper side 24 of the fork tines 61, 62 would collide with the pallet 12 when inserting it. The first auxiliary marking 22a indicates the position of the upper side 24 of the fork tines 61, 62 if the angle of inclination were zero degrees, unlike the actual case, i.e. if the upper side 24 of the fork tines 61, 62 were at least approximately horizontal. Based on the deviation between the first auxiliary marking 22a and the first main marking 20a, the driver of the industrial truck 2 is able to determine how steeply the fork 4 is tilted.

[0039] The processing unit 10 is further configured to receive data relating to a steering angle of the industrial truck 2. Based on this data, the second auxiliary marking 22b is calculated. This indicates a future deviation of the center between the forks 61, 62 when traveling in the conveying plane with the set steering angle. In other words, the second auxiliary marking 22b indicates where the second main marking 20b will soon be when traveling with the set steering angle. This information indicates to the driver of the industrial truck 2 that they are currently at a steering angle that deviates from straight-ahead travel.Furthermore, the marking 22b allows the driver, for example, to aim at the middle pallet block 14b using the second auxiliary marking 22b, so that a successful penetration into the load-carrying areas 16a, 16b of the pallet 12 takes place after the industrial truck 2 has completed its journey.

[0040] Fig. 4shows the image of the fork camera 8, 8' displayed on the monitor, with the main markings 20a, 20b and the auxiliary markings 22a, 22b aligned. In other words, this also means that the fork 4 of the industrial truck 2 is aligned horizontally and the steering angle for straight-ahead travel is identically zero. The main markings 20a, 20b and the auxiliary markings 22a, 22b can be distinguished in the display on the monitor by color or structural features. For example, the main markings 20a, 20b are shown with a solid line and the auxiliary markings 22a, 22b with a dashed line. However, different colors, line thicknesses, brightness, etc. can also be provided for the corresponding markings 20a, 20b or 22a, 22b.

[0041] According to a further embodiment, it is provided that image data of a load to be picked up, for example a pallet 12, are present in the captured image data of the fork camera 8, 8', wherein the processing unit 10 identifies load-carrying areas 16a, 16b of this load to be picked up.

[0042] Fig. 5shows an image from the fork camera 8, 8' displayed on the monitor, with the industrial truck 2 approaching, for example, a schematically indicated rack 26 with an exemplary compartment with a standard shelf 28. The third main marking 20c shows, for example, the outer edges of the second fork 62. Based on the load-carrying areas 16a, 16b detected in the rack 26 (indicated by a dashed line), target positions 30a, 30b of the tips 71, 72 of the fork tines 61, 62 are calculated when the fork tines 61, 62 insert into the load-carrying areas 16a, 16b. These target positions 30a, 30b, for example, colored squares, are superimposed on the image data from the fork camera 8, 8' and displayed on the monitor. As the distance of the fork 4 decreases, for example when traveling towards the shelf 26, these target positions 30a, 30b are dynamically adjusted.The driver of the industrial truck 2 can successively align or hold the tips 71, 72 of the forks 61, 62 with these target positions 30a, 30b, so that, for example, safe and reliable pallet insertion is ensured.

[0043] In particular, it is provided that the fork camera 8, 8' captures live image data, and the processing unit 10 is also configured to perform real-time data processing. Thus, a live image with markings 18 calculated in real time is displayed on the monitor.

[0044] The industrial truck 2 shown is, for example, a forklift truck and the load to be picked up is, for example, a pallet 12. However, the system described above can also be used in a corresponding manner without restrictions for other industrial trucks or other loads to be picked up.

[0045] All mentioned features, including those that can be gathered from the drawings alone, as well as individual features disclosed in combination with other features, are considered essential to the invention, both individually and in combination. Embodiments according to the invention may be fulfilled by individual features or a combination of several features. Within the scope of the invention, features marked "in particular" or "preferably" are to be understood as optional features. List of reference symbols

[0046] 2Industrial truck 4Fork 8, 8'Fork camera 10Processing unit 12Pallet 14a - 14cPallet blocks 16a, 16bLoad handling areas 18Marking 20aFirst main marking 20bSecond main marking 20cThird main marking 22aFirst auxiliary marking 22bSecond auxiliary marking 24Top 26Shelf 28Shelf base 30a, 30bTarget positions 61, 62Forks 71, 72Tips

Claims

1. A method for operating an industrial truck (2) having a fork (4) comprising at least two fork arms (61, 62), a fork arm camera (8, 8'), a monitor, and a processing unit (10) for reading and processing the image data captured by the fork arm camera (8, 8') and for producing and transmitting processed image data shown on the monitor, wherein the fork arm camera (8, 8') is disposed on the fork (4) such that at least part of a tip (71, 72) of at least one of the two fork arms (61, 62) is present in the field of view of the fork arm camera (8, 8'), characterized in that the processing unit (10) performs the following steps: capturing the at least one tip (71, 72) of the at least one fork arm (61, 62) in the captured image data, calculating at least one marking (18) indicating a position of the fork (4), and reproducing on the monitor the image data captured by the fork arm camera (8, 8') superimposed with the marking (18).

2. The method according to claim 1, characterized in that the marking (18) indicates the top side (24) and / or the outer edges of the fork arms (61, 62) and / or the center between the fork (61, 62).

3. The method according to claim 1 or 2, characterized in that the processing unit (10) further receives data relating to an angle of inclination of the fork (4) and calculates on the basis of said data an auxiliary marking (22a) indicating a deviation of the position of the fork (4) from a zero-degree position of the fork (4), wherein image data captured by the fork arm camera (8, 8') and superimposed with the auxiliary marking (22a) is reproduced on the monitor.

4. The method according to any one of the claims 1 to 3, characterized in that the processing unit (10) further receives data relating to a steering movement angle and calculates on the basis of said data a further auxiliary marking (22b) indicating a future deviation of the position of the fork (4) from the current position thereof in a plane of transport for assumed travel at the steering movement angle, wherein image data captured by the fork arm camera (8, 8') and superimposed with the further auxiliary marking (22b) is reproduced on the monitor.

5. The method according to any one of the claims 1 to 4, characterized in that a load to be taken up having at least two visible load support areas (16a, 16b) is present in the captured image data, wherein the target positions (30a, 30b) of the tips (71, 72) of the fork arms (61, 62) are calculated when inserting the fork arms (61, 62) into the visible load support areas (16a, 16b), and image data captured by the fork arm camera (8, 8') and superimposed with the calculated target position (30a, 30b) of the tips (71, 72) of the fork arms (61, 62) is displayed on the monitor.

6. The method according to any one of the claims 1 to 5, characterized in that live image data is captured by the fork arm camera (8, 8'), the processing unit (10) performs the indicated steps in real time, and a superposition of a live image and the marking (18) calculated in real time are reproduced on the monitor.

7. An industrial truck (2) having an assistance system and having a fork (4) comprising at least two fork arms (61, 62), wherein the assistance system comprises at least one fork arm camera (8, 8'), a monitor, and a processing unit (10), wherein the fork arm camera (8, 8') is disposed on the fork (4) such that at least part of a tip (71, 72) of at least one of the two fork arms (61, 62) is present in the field of view of the fork arm camera (8, 8'), characterized in that the processing unit (10) is set up for: capturing the at least one tip (71, 72) of the at least one fork arm (61, 62) in the image data, calculating at least one marking (18) indicating a position of the fork (4), and reproducing on the monitor the image data captured by the fork arm camera (8, 8') and superimposed with the marking (18).

8. The industrial truck (2) according to claim 7, characterized in that the marking (18) indicates the top sides (24) and / or the outer edges of the fork arms (61, 62) and / or the center between the fork (61, 62).

9. The industrial truck (2) according to claim 7 or 8, characterized in that the processing unit (10) is further set up for receiving data relating to the angle of inclination of the fork (4), and for calculating on the basis of said data an auxiliary marking (22a) indicating a deviation of the position of the fork (4) from a zero position of the fork (4), and for displaying on the monitor image data captured by the fork arm camera (8, 8') and superimposed with the auxiliary marking (22a).

10. The industrial truck (2) according to any one of the claims 7 to 9, characterized in that the processing unit (10) is further set up for receiving data relating to a steering movement angle and for calculating on the basis of said data an additional auxiliary marking (22b) indicating a future deviation of the position of the fork (4) from the current position thereof in a plane of transport for assumed travel at the steering movement angle, and for reproducing on the monitor image data captured by the fork arm camera (8, 8') and superimposed with the additional auxiliary marking (22b).

11. The industrial truck (2) according to any one of the claims 7 to 10, characterized in that a load to be taken up having at least two visible load support areas (16a, 16b) is present in the captured image data, wherein the processing unit (10) is further set up for calculating target positions (30a, 30b) of the tips (71, 72) of the fork arms (61, 62) when inserting the fork arms (61, 62) into the visible load support areas (16a, 16b), and image data captured by the fork arm camera (8, 8') and superimposed with the calculated target position (30a, 30b) of the tips (71, 72) of the fork arms (61, 62) is displayed on the monitor.

12. The industrial truck (2) according to any one of the claims 7 to 11, characterized in that the fork arm camera (8, 8') is set up for capturing live image data and the processing unit (10) is set up for performing the steps indicated in real time and for reproducing on the monitor a superposition of a live image and the marking (18) calculated in real time.

Images