Method and device for collision avoidance during operation of an industrial truck

Description

[0001] The invention relates to a method for collision avoidance during operation of an industrial truck, wherein the industrial truck has a camera that records a monitoring area and supplies the image data thus determined to an image processing device, which creates action instructions and / or control commands for the operation of the industrial truck by evaluating the image data, as well as to an appropriately equipped industrial truck.

[0002] Industrial trucks used by industrial and commercial companies must meet ever-increasing demands in terms of speed and efficiency. Therefore, automated or semi-automated industrial trucks are increasingly being used, where the driver can delegate at least some of the control tasks to the truck's technology and can primarily focus on picking up and dropping off goods, for example, in a warehouse. Autonomous industrial trucks are also already in use; they move driverless through warehouses, for example, and automatically navigate to positions for picking up and dropping off goods.

[0003] In all of these cases, collision avoidance systems are required to ensure safe operation of the industrial trucks and to prevent unintentional collision with an obstacle or even injury to a person on the truck's path.

[0004] For example, EP 1 728 757 B1 discloses an industrial truck that uses collision avoidance or collision warning systems. These systems are based on contactless distance measurement, particularly using electromagnetic waves and / or ultrasound. A laser unit and / or a radar unit is preferably used for this purpose. Three-dimensional objects can be detected using laser radiation, for example.

[0005] EP 2 851 331 A1 describes a method for controlling an order-picking industrial truck. Order-picking industrial trucks are frequently used as a type of industrial truck for order picking, i.e., assembling deliveries of goods in a warehouse. The order-picking industrial truck, with a load carrier mounted on forks, such as a pallet or a wire mesh basket, is moved through the rack aisles of a warehouse in which the goods to be picked or picked are stored. Depending on the order, the picker or the person carrying out the order removes the goods from the shelf, places them on or in the order-picking industrial truck's load carrier, and must then travel to the next pick-up location within the warehouse or rack aisle of the warehouse for the next item.Once the respective picking order has been fully processed, the order picker drives the picking industrial truck to a goods drop-off point. The large number of required movements of the industrial truck that the order picker must perform requires a considerable amount of time, as the order picker must walk to a steering and control unit on the picking industrial truck to operate the relevant travel switches and steer the vehicle during the movement, and then return to the position to pick up the goods. Therefore, this publication proposes an automatic tracking system for the industrial truck. An optical sensor is provided for this purpose, which detects a specific monitoring area. If the order picker exceeds a predetermined limit within the monitoring area, the industrial truck is automatically tracked.The industrial truck automatically follows the movements of the order picker, for example, as they walk along a row of containers with the goods to be picked or along a rack aisle and remove these goods. Whenever the order picker moves along the rack aisle, the industrial truck automatically follows them. The optical sensor can be designed as a camera, and the position of a person being followed can be detected using image processing methods. The industrial truck can also be equipped with a personnel protection device, so that autonomous operation is also possible. The personnel protection device monitors a close area for collisions with obstacles or people. It can be based on monitoring at a low height above the ground, for example using laser scanners that scan an area approximately 150 mm above the ground.The data obtained by the optical sensor, especially if it is three-dimensional data, can be used not only for tracking the person and determining a target lane but also generally for vehicle guidance, for example in the sense of an autonomously driving vehicle.

[0006] Common collision avoidance methods are based on distance measurement using laser, ultrasound, or radar. This method determines the direct clearance in the direction of travel of the industrial truck. Collision avoidance using laser scanners requires a significant investment. A laser scanner consists of a laser diode and a receiver located in a motor-driven, rotating head, as well as a high-resolution incremental encoder for angle measurement. The laser light incident on an obstacle is reflected back into itself, i.e., to the transmitter, or to a receiver mounted directly next to it. Since laser scanners and laser sensors are precision devices, a complex evaluation computer is required to evaluate the three-dimensional measurement data, and the precise calibration of the laser scanner requires considerable equipment, this method is very expensive.On the other hand, the measurement accuracy of radar and ultrasound methods is not high enough to allow for precise responses. Camera-based systems have so far been used primarily for navigation. For this purpose, pairs of cameras are typically connected to form stereo camera systems that capture a three-dimensional image of the entire surrounding space. This generates very large amounts of data, the processing of which requires considerable computing power.

[0007] From US 9 358 975 B1 a generic method is known in which an industrial truck has one or more cameras that record the surroundings and with which distances to objects in the surroundings can be determined.

[0008] US 2016 / 0313740 A1 discloses an autonomous industrial truck that navigates using markings on the roadway. The truck is equipped with a downward-facing camera to detect the markings on the ground.

[0009] EP 2 339 376 A1 discloses a method for collision avoidance during the operation of an industrial truck. The industrial truck has a sensor that detects a monitoring area and delivers the resulting sensor data to a processing device, which evaluates the sensor data to generate instructions and / or control commands for the operation of the industrial truck. The sensor is designed as a laser scanner that emits a horizontal light beam. If there are objects on the travel surface, the emitted light beam is reflected, and the direction / angular position (angular position of a deflection unit of the sensor, which is detected using the signal from an encoder) and the distance (by measuring the time of flight of the light beam) of the object are determined from the reflected light beam.

[0010] US 2011 / 093134 A1 discloses a system for collision avoidance during the operation of industrial trucks, in which the respective industrial trucks send their position data to a higher-level collision avoidance system, which calculates a trajectory of each industrial truck from the position data and determines collision areas from the trajectories of the industrial trucks, and in the event that a collision between two industrial trucks is predicted in a collision area, sends corresponding instructions to the industrial trucks.

[0011] DE 10 2014 209 338 A1 discloses a time-of-flight camera system for free-field detection in the environment of a work machine.

[0012] EP 2 555 014 A1 discloses a method for detecting planes with respect to possible interfering objects by a sensor of a vehicle.

[0013] The present invention is based on the object of providing a method for collision avoidance in an industrial truck which ensures safe collision avoidance during the operation of industrial trucks with low investment expenditure.

[0014] This object is achieved according to the invention in that the camera is directed towards the running floor located in front of the industrial truck in the direction of travel and detects a monitoring area located in front of the industrial truck in the direction of travel and the camera detects a running floor located in the monitoring area, the image processing device detects the running floor as a coherent area from the image data determined and, upon detection of objects that stand out from this running floor, creates the action instructions and / or control commands to avoid collisions.

[0015] The invention is based on the idea that in typical industrial truck applications, e.g., warehouses, flat floors with a homogeneous texture and color can be assumed. The invention is based on the realization that a simplified collision avoidance method can be applied in such an environment. The core idea is to create a method that can distinguish between the floor and "non-floor" (obstacles). For this purpose, the use of a camera that captures the traveling floor is proposed. The camera's image data is sent to an image processing device, which recognizes the traveling floor as a coherent area. If the camera detects any object that visually stands out from the homogeneous traveling floor, the image processing device can react to it.The object can be a three-dimensional obstacle, such as an object or a person, or simply a disruption of the homogeneity of the floor caused by a different color. In the latter case, this area can be monitored and, if necessary, a warning signal can be issued. Since the invention does not require the entire surrounding area of ​​the industrial truck to be monitored, but only distinguishes between ground and "non-ground," the technical complexity is significantly lower than with conventional camera-based systems used for 3D navigation.

[0016] Image processing is preferably used to segment and classify the road surface. If areas with structural and / or color deviations are detected, these are monitored, and if a specified deviation level is exceeded, a warning is issued.

[0017] According to the invention, odometric data from the industrial truck is recorded during travel and transmitted to the image processing system in addition to the image data. Odometric data refers, in particular, to the distance traveled, resulting from the number of wheel revolutions of the industrial truck. Stereo images are generated from the images taken sequentially during travel, the baseline of which represents the travel distance determined from the odometric data.

[0018] The image processing system advantageously calculates distance information for each point in the combined stereo image from the determined travel distance. This allows the camera to not only detect whether other objects or people stand out visually from the homogeneous travel surface, but also calculate their distance. The dimensions of the obstacle can also be calculated in this way.

[0019] According to a particularly preferred embodiment of the invention, the image processing device calculates a three-dimensional image from the stereo image data and the distance information. An object identified as an obstacle in the three-dimensional image by the image processing device is expediently evaluated according to its three-dimensional dimensions, so that situation-appropriate action instructions or control commands for collision avoidance can be generated. A warning, a stop, or evasive action can be initiated.

[0020] The invention also relates to an industrial truck with a collision avoidance device comprising a camera and an image processing device operatively connected thereto, which carries out a method according to the invention.

[0021] Preferably, the industrial truck has a device for recording odometric data of the industrial truck, which is also operatively connected to the image processing device.

[0022] According to an advantageous development of the inventive concept, the image processing device is designed to computationally generate three-dimensional representations from the images recorded by the camera and the odometric data.

[0023] The invention provides a technically elegant solution for collision avoidance, which enables safe manual, semi-automated or autonomous operation of an industrial truck with low investment costs, particularly in areas of application with a substantially homogeneous running floor, e.g. in warehouses.

[0024] Compared to laser systems, the solution according to the invention has the advantage that a simple camera can be used instead of an expensive laser scanner. Furthermore, the complex calibration process is eliminated.

[0025] Compared to radar or ultrasound systems, the invention offers the advantage of better resolution, allowing a more precise response to obstacles.

[0026] Finally, the invention also offers decisive advantages over conventional camera-based navigation systems, which may also offer collision avoidance. Such systems typically use stereo cameras to capture three-dimensional images of the entire surrounding space. This requires a correspondingly large amount of computing power to process the enormous amounts of data. The invention, in contrast, enables the use of a simple monocular camera system to monitor a much smaller space (in the area of ​​the driving floor) with correspondingly lower data processing effort.

[0027] Further advantages and details of the invention are explained in more detail with reference to the embodiment shown in the schematic figures. Figure 1 shows an industrial truck with a camera, Figure 2 shows an industrial truck with a camera and a segmented floor, Figure 3 shows the segmentation of a floor, Figure 4 shows the segmentation of "non-floor", Figure 5 shows an illustration of observing an obstacle and Figure 6 shows the stereo effect when detecting an obstacle.

[0028] In the examples of Figures 1, 2 , 5 and 6 The industrial truck is a forklift truck 1 with a camera 2 located on the floor 3 (moving floor) of a warehouse. Since warehouses are assumed to have a flat floor with a very homogeneous structure and color, it is technically simple and cost-effective to use the camera 2 and a downstream image processing device (not shown in the figure) to detect whether objects or people visually stand out from this floor 3.

[0029] In Figure 1It is shown how the camera 2 is aligned with the floor (running floor) in front of the industrial truck 1 in the direction of travel, so that the image processing system can recognize the floor as a coherent area.

[0030] In the example of Figure 2 The ground 3 is detected, segmented and classified by the image processing system. Figure 3 shown how the camera 2 captures the segmented floor 3.

[0031] Figure 3 shows the segmentation of the floor 3 from Figure 2 in detail. Two camera images are shown as examples, which were taken one after the other while the industrial truck was traveling along the segmented floor. The left image from Figure 3 was taken first, followed by the right one. Camera 2 of the industrial truck 1 from Figure 2 sees while driving along the segmented floor 3, so to speak, first the left image Figure 3and then the right one. If the area captured by camera 2 is interrupted due to color changes, this is detected by the image processing system. These objects are then closely monitored. The driver of industrial truck 1 can be warned.

[0032] In Figure 4 is like in Figure 3 The segmentation of the floor 3 is shown. In this case, the camera 2 in the right-hand image detects an object 4 that stands out from the homogeneous floor 3. This object 4 is recognized by the image processing device as a supposed obstacle 4.

[0033] In Figure 5 It is shown how the camera 2 of the industrial truck 1 observes the supposed obstacle 4 lying on the segmented floor 3.

[0034] Figure 6 shows the industrial truck 1 from Figure 5in a position after having moved a travel distance dx in the direction of travel. The image processing device observes prominent points on the perceived obstacle 4 via camera 2 and, with knowledge of the traveled distance dx and the stereo effect from the images taken consecutively during the journey, can determine the necessary 3D information about the perceived obstacle 4. This 3D information can be processed by the image processing device, allowing a targeted response to the situation. A warning can be issued, a stop can be initiated, or the vehicle can be evaded.

Claims

1. Method for collision avoidance during the operation of an industrial truck (1), wherein the industrial truck (1) comprises a camera (2), which scans a monitoring region and transmits image data thus captured to an image processing apparatus which, by the evaluation of image data, generates instructions and / or control commands for the operation of the industrial truck (1), characterized in that the camera (2) is oriented in the direction of travel towards a surface of travel (3) ahead of the industrial truck (1), and scans a monitoring region ahead of the industrial truck (1) in the direction of travel, the camera (2) scans a surface of travel (3) which is located in the monitoring region, the image processing apparatus, from image data thus captured, identifies the surface of travel (3) as a coherent region and, in the event of the detection of objects (4) which contrast from the surface of travel (3), generates instructions and / or control commands for collision avoidance, whereas during the travel of an industrial truck (1), odometry data for the industrial truck (1) are captured and transmitted to the image processing apparatus, which generates stereo images from images which are captured in a temporally sequential manner during travel, the base line of which represents the distance of travel (dx) determined from odometry data.

2. Method according to Claim 1, characterized in that image processing involves segmentation and classification of the surface of travel (3) and, upon the detection of regions featuring deviations in structure and / or coloration, these regions are monitored and, in the event that a predefined degree of deviation is exceeded, a warning instruction is generated.

3. Method according to Claim 1 or 2, characterized in that the image processing apparatus, from the distance of travel (dx) thus determined, the image processing apparatus calculates distance information at each point in a common stereo image.

4. Method according to Claim 3, characterized in that the image processing apparatus generates a three-dimensional image from stereo image data and distance information.

5. Method according to Claim 4, characterized in that the image processing apparatus evaluates an object (4) in the three-dimensional image which is identified as an obstacle by reference to its three-dimensional measurements, and generates situationally appropriate instructions or control commands for collision avoidance.

6. Industrial truck having a device for collision avoidance, which comprises a camera and an image processing apparatus which is operatively connected thereto, and which executes a method according to one of Claims 1 to 5.

7. Industrial truck according to Claim 6, characterized in that the industrial truck (1) comprises an apparatus for the capture of odometry data for the industrial truck (1), which is also operatively connected to the image processing apparatus.

8. Industrial truck according to Claim 7, characterized in that the image processing apparatus is configured for the computerized generation of three-dimensional representations from images which are captured by the camera and from odometry data.