Method and control device for controlling an industrial truck

Abstract

The invention describes a method for controlling an industrial truck in a warehouse, comprising the steps of: reading in (S1) a point cloud of the warehouse in an area surrounding the industrial truck, wherein the point cloud that has been read in has point information relating to at least one object in the surrounding area, converting (S3) the point information into a frequency distribution of the point information, determining (S4) a statistical distribution parameter of the point information in the frequency distribution, classifying (S5) the object as a person or a warehousing item on the basis of the statistical distribution parameter, and outputting (S6) a control signal for controlling the industrial truck in the warehouse on the basis of a classification resulting from the classifying step (S5). The invention also describes a control device for carrying out the method and an industrial truck comprising a control device of this type.

Description

[0001] Method and control device for controlling an industrial truck

[0002] Technical area

[0003] The present invention relates to a method and a control device for controlling an industrial truck in a warehouse. The present invention also relates to an industrial truck with such a control device.

[0004] State of the art

[0005] It is known from the prior art to control a vehicle based on environmental detection. For example, DE 10 2018 117 428 A1 discloses detecting objects in the vehicle's environment and classifying them based on their contours. Furthermore, WO 2020 / 049154 A1 discloses extracting sensor-independent features to perform object classification based on a neural network.

[0006] Description of the invention

[0007] In one aspect, the present invention relates to a method for controlling an industrial truck in a warehouse. The industrial truck can be a conveying means for transporting goods. The industrial truck can be an industrial truck or an industrial conveyor. For example, the industrial truck is a forklift. The warehouse can have a storage facility in a production or logistics facility. Goods to be transported by the industrial truck can be stored in the warehouse.

[0008] The method comprises, as one step, reading in a point cloud of the warehouse in an environment of the industrial truck. The method can comprise, as a further step, detecting the point cloud with an environment detection sensor. The environment detection sensor can be arranged on the industrial truck to detect the point cloud. The method can comprise, as a further step, outputting a control signal for detecting the point cloud with the environment detection sensor. The environment detection sensor can comprise a distance-measuring sensor. Alternatively or additionally, the environment detection sensor can comprise an image-capturing sensor. The distance-measuring sensor can be embodied as a scanning sensor. The distance-measuring sensor can be, for example, a laser scanner, a radar measuring device, or an ultrasonic measuring device.The laser scanner, radar measuring device or ultrasonic measuring device can be designed as a two-dimensional or three-dimensional scanning sensor.

[0009] If the environment detection sensor system has a distance-measuring sensor, the warehouse in the vicinity of the industrial truck can be detected or scanned at specific points. The distance-measuring sensor can be used to record the point cloud of the warehouse in the vicinity of the industrial truck. The point cloud can be a two-dimensional or three-dimensional point cloud. The image-capturing sensor can be designed as a camera system. The camera system can have an RGB camera. The camera system can have a stereo camera system. If the environment detection sensor system has an image-capturing sensor as an alternative or in addition to a distance-measuring sensor, an image of the warehouse in the vicinity of the industrial truck can be detected or recorded. The image-capturing sensor can record a point cloud with image points of the warehouse.

[0010] The imported point cloud contains point information about at least one object in the surrounding area. The point information can contain the measurement points acquired with the distance-measuring sensor. Alternatively or additionally, the point information can contain the image points acquired with the image-capturing sensor. The point information can contain meta-information about the measurement points. Alternatively or additionally, the point information can contain meta-information about the image points. The point information can contain at least one point coordinate of the measurement points. Alternatively or additionally, the point information can contain at least one piece of color information in a visible or non-visible spectral range of the image points.

[0011] The method comprises a further step of converting the point information into a frequency distribution of the point information. Converting the point information can comprise transforming the point information from a sensor-related coordinate system into a coordinate system of the frequency distribution. The frequency distribution can be a one-dimensional or multi-dimensional frequency distribution. The frequency distribution can be a statistical function that specifies how frequently the point information occurs. Thus, a subset of the imported point cloud can be converted into the frequency distribution. In the step of converting the point information, a multi-dimensional point cloud can thus be converted into a one-dimensional frequency distribution of the point information. For this purpose, the point information can form a one-dimensional subset of the point cloud.The point information can be derived from the point cloud as a one-dimensional subset. Subsequent steps of the process can thus be performed particularly efficiently based on the transferred frequency distribution of the point information.

[0012] The method comprises, as a further step, determining a statistical distribution parameter of the point information in the frequency distribution. The statistical distribution parameter can be a dispersion of the point information in the frequency distribution. For example, the statistical distribution parameter can be a standard deviation or variance of the point information in the frequency distribution. The statistical distribution parameter can thus be used to determine an inhomogeneity of the point information.

[0013] The method comprises, as a further step, classifying the object into a human or a warehouse object based on the statistical distribution parameter. The object can be an obstacle object for the industrial truck or a work object for the industrial truck. The industrial truck can be controlled using the method in such a way that a collision with the obstacle is avoided. The industrial truck can also be controlled using the method in such a way that the approach of the industrial truck to the work object is improved. According to one embodiment, the classes for classifying the object consist of the human and the warehouse object.

[0014] In the classification step, the object can be classified as a human or a storage object depending on a threshold value for the statistical distribution parameter. If the threshold value is exceeded, the object can be classified as a human. If the threshold value is undershot, the object can be classified as a storage object. Classifying the object based on the statistical distribution parameter is based on the realization that point information relating to the human can be more inhomogeneous than point information relating to the storage object. This inhomogeneity can result in a corresponding statistical distribution parameter, for example, a greater scatter of the point information, in the frequency distribution.

[0015] The method comprises, as a further step, outputting a control signal for controlling the industrial truck in the warehouse based on a classification resulting from the classifying step. According to one embodiment, the resulting classifications consist of the classification into the human and the classification into the warehouse object. The control signal can be output depending on whether the object is classified into the human or the warehouse object.

[0016] The invention thus allows classification into a person or a warehouse object based on a statistical distribution of point information, in order to automatically control the industrial truck depending on the person or warehouse object identified by the classification. The statistical distribution parameter can differ significantly depending on whether the point information relates to a person or a warehouse object. The steps of classifying and controlling the industrial truck can thus be carried out reliably.

[0017] According to one embodiment of the method, the read-in point cloud can contain point information from at least two objects located in the surrounding area. The method can comprise, as a further step, segmenting the read-in point cloud into at least one segment. The segment can contain point information assigned to an object of the at least two objects from the point information of the read-in point cloud. The segmentation can be carried out based on the point information. A segment can thus contain point information that relates to only one object. In the transfer step, the assigned point information can be transferred into the frequency distribution. The frequency distribution and the determined statistical distribution parameter can therefore relate to a segmented object, which can be a person or a storage object.

[0018] The warehouse object can, for example, comprise or be a shelf or a pallet. The warehouse object can also be an object of the warehouse infrastructure. The person can be a pedestrian in the warehouse. The steps of the method can thus be performed for a large number of objects located in the surrounding area. The method can therefore be carried out reliably even if there are people and warehouse objects in the vicinity of the industrial truck.

[0019] According to a further embodiment of the method, in the step of determining the statistical distribution parameter, the distribution parameter can be determined based on a dispersion parameter in the frequency distribution. The dispersion parameter can be the standard deviation or the variance of the point information in the frequency distribution. The step of classifying the object can thus be carried out depending on the dispersion parameter. If the dispersion parameter exceeds a predetermined threshold for the dispersion parameter, the object can be classified as a human. If the dispersion parameter falls below the predetermined threshold, the object can be classified as a storage item. The classification can thus be carried out efficiently based on a dispersion of the point information.

[0020] According to a further embodiment of the method, in the transferring step, the point information can be transferred into a histogram of the point information. The point information can be represented in the histogram as a map of the point information. In the step of determining the statistical distribution parameter, the distribution parameter can be determined in the histogram or in a sub-region of the histogram. The distribution parameter can be determined based on a scatter parameter in the histogram. The statistical distribution parameter can be determined functionally or image-based in the histogram. The distribution parameter can thus be determined particularly efficiently using image processing methods in the map of the point information in the histogram. The method can therefore advantageously also be carried out without a KL approach.

[0021] According to a further embodiment, the point information of the read-in point cloud can include spatial point coordinates of the at least one object located in the surroundings. The spatial point coordinates can be at least one-dimensional point coordinates of the at least one object located in the surroundings. In the conversion step, the spatial point coordinates can be converted into the frequency distribution. The determined statistical distribution parameter or the dispersion parameter can thus define a dispersion of the spatial point coordinates of the object. The described classification can thus be performed depending on the spatial extent of the object in at least one dimension.

[0022] According to a further embodiment of the method, the point information of the read-in point cloud can include signal intensities of measurement signals reflected by the at least one object located in the surrounding area for capturing the point cloud. In the conversion step, the signal intensities can be converted into the frequency distribution. The determined statistical distribution parameter or the scatter parameter can therefore define a scatter of the signal intensities of measurement signals reflected by the object. The classification can thus be performed depending on material properties or a geometry of the object, which can influence the signal intensities.

[0023] According to a further embodiment of the method, the point information of the read-in point cloud can contain spatial velocity coordinates of the at least one object located in the surroundings. If the surroundings detection sensor system includes the radar device, the spatial velocity coordinates can be determined from measurement data of the radar device. In the conversion step, the spatial velocity coordinates can be converted into the frequency distribution. The determined statistical distribution parameter or the dispersion parameter can define a dispersion of the spatial velocity coordinates. The classification can thus be performed depending on whether areas of the object are moving relative to one another or whether the object is a rigid object.

[0024] According to a further embodiment of the method, in the step of outputting the control signal, a control signal for interrupting a work task performed by the industrial truck can be output to a work device of the industrial truck. The work device can be a drive device for driving the industrial truck. The work device can also be a transport device or lifting device of the industrial truck for transporting or lifting goods. The control signal for interrupting the work task can be output if the classification step results in the object being classified as a person. Movement of the industrial truck while performing the work task can thus be stopped based on the control signal in order to ensure the work safety of the person.

[0025] According to a further embodiment of the method, in the step of outputting the control signal, a control signal for controlling or limiting the driving dynamics of the industrial truck can be output. The control signal can be output to limit the longitudinal dynamics of the industrial truck. A speed or acceleration of the industrial truck can thus be limited. The control signal can also be output to decelerate the industrial truck. The control signal can be output to limit the lateral dynamics of the industrial truck. A steering angle of the industrial truck can thus be limited. The control signal for limiting the driving dynamics can be output if the classification result from the classifying step is that the object is classified as a warehouse item. Alternatively, the control signal can also be output if the classification result from the classifying step is that the object is classified as a person.

[0026] According to a further embodiment of the method, the step of classifying the object can be performed based on a machine learning model. The machine learning model can comprise a KL approach or a neural network. According to this embodiment, at least one of the frequency distribution and the statistical distribution parameter can be read into the machine learning model. Thus, advantageously, only the frequency distribution or the statistical distribution parameter can form an input variable for the machine learning model.

[0027] According to a further embodiment of the method, the step of classifying the object can be performed based on the machine learning model, with the histogram being read into the machine learning model. Thus, a histogram-based classification of the object can be performed particularly efficiently based on the histogram. Image-based evaluation of the histogram is particularly advantageous in this case.

[0028] In a further aspect, the present invention relates to a control device for controlling an industrial truck in a warehouse. The control device has an interface for reading in a point cloud of the warehouse in an environment of the industrial truck. The read-in point cloud contains point information from at least one object located in the environment.

[0029] The control device is configured to convert the point information into a frequency distribution of the point information. The control device is also configured to determine a statistical distribution parameter of the point information in the frequency distribution. The control device is also configured to classify the object into a human or a warehouse object based on the statistical distribution parameter. The control device also has an interface for outputting a control signal for controlling the industrial truck in the warehouse based on the resulting classification.

[0030] The control device can be configured to carry out the method according to the preceding aspect. The control device can be arranged on the industrial truck. Features and embodiments of the method according to the preceding aspect can be corresponding features and embodiments of the control device, wherein the control device can be configured to carry out each of the steps described for the method. The control device can have corresponding units for carrying out the method steps.

[0031] In a further aspect, the present invention relates to an industrial truck. The industrial truck can be designed as described in the preceding aspects. The industrial truck has a control device for controlling the industrial truck according to the preceding aspect.

[0032] Short description of the characters

[0033] Figure 1 shows an industrial truck and a control device for controlling the industrial truck according to respective embodiments of the invention.

[0034] Figure 2 shows schematically the industrial truck in a warehouse to explain the invention.

[0035] Figure 3 shows a flowchart with method steps for carrying out a method for controlling the industrial truck according to an embodiment of the invention.

[0036] Detailed description of embodiments

[0037] Figure 1 shows an industrial truck 100 having a work device 30 for performing a work task with the industrial truck 100. An environment detection sensor system 20 is arranged on the industrial truck 100, with which a point cloud is detected in the environment 4 of the industrial truck 100. The point cloud comprises preprocessed measurement data from the environment detection sensor system 20. The point cloud comprises points relating to an object 10 located in the environment 4 of the industrial truck 100 and shown in Figure 2.

[0038] The industrial truck 100 has a control device 110 for controlling the work device 30. The point cloud captured by the environment detection sensor system 20 is read in by the control device 110 via an interface 112 and processed in order to control the work device 30 based on the processed point information. According to one embodiment, the point cloud includes the point information read in by the control device 110. According to a further embodiment, the point information is derived from the read-in point cloud by the control device 110. The control device 110 outputs a control signal via an interface 114 for controlling the work device 30 based on the point information.

[0039] Figure 2 shows the industrial truck 100 in a warehouse 2. In the warehouse 2, objects 10 are located in the environment 4 of the industrial truck 100. The point cloud recorded in the environment 4 contains measurement data or points related to the objects 10. One object 10 of the objects 10 is a person 12 located in the warehouse 2, and another object 10 of the objects 10 is a storage item 14 located in the warehouse 2, according to one embodiment, a shelf.

[0040] Figure 3 shows method steps S1 to S6 for controlling the industrial truck 100, which are carried out by the control device 110.

[0041] In a first step S1, the control device 110 reads in the point cloud via the interface 112. The read-in point cloud contains point information about the objects 10 located in the environment 4. According to one embodiment, the point information comprises point coordinates. According to another embodiment, the point information comprises signal intensities of measurement signals from the environment detection sensor system 20 reflected by the objects 10.

[0042] In a second step S2, the control device 110 segments the point cloud or the point information. The read-in point cloud is segmented into at least two segments. A first segment has point information assigned to one object 10 of the objects 10. A second segment has point information assigned to the further object 10 of the objects 10. Point information relating to different objects 10 can thus be separated using the segmentation step. Subsequent steps of the method are carried out for the respectively assigned point information. In a third step S3, the control device 110 converts the point information into a frequency distribution. The segmented point information is converted into a histogram containing the frequencies of the segmented point information.According to corresponding embodiments, the histogram has frequencies of the point coordinates and the signal intensities.

[0043] In a fourth step S4, the control device 110 determines a distribution parameter in the frequency distribution. The distribution parameter is a dispersion parameter of the frequencies of the histogram. The histogram forms an input variable for a machine learning model for classifying the object 10 as a human 12 or a storage object 14 based on a dispersion of the segmented point information. According to one embodiment, the dispersion parameter forms a classification variable in the machine learning model.

[0044] In a fifth step S5, the control device 110 classifies the object 10 into a human 12 or a storage object 14. Classification into the human 12 or the storage object 14 is an output variable of the machine learning model, which is based on the scatter parameter according to the described embodiment. The machine learning model is based on the assumption that point coordinates or signal intensities assigned to the human 12 scatter more strongly than point coordinates or signal intensities assigned to the storage object 14. The greater scatter of the point coordinates for a storage object 14 is based on the assumption that the storage object 14 is a flatter or less curved object 10, at least with respect to one coordinate axis and compared to the human 12.The greater scattering of the signal intensities for a human 12 is based on the assumption that, compared to the warehouse object 14, the human 12 has an object surface detectable by the environment detection sensor system 20, which surface has more materials and curvatures, resulting in more inhomogeneous and therefore more scattered signal intensities of the measurement signals reflected from the object surface. In a sixth step S6, the control device 110 outputs the control signal via the interface 114 to the work device 30. If a classification from the previous step S5 results in a human 12 being located in the environment 4 of the industrial truck 100, a control signal is output to the work device 30 as a control signal, with which the execution of the work task is interrupted.If a classification from the previous step S5 results in a warehouse object 14 being located in the environment 4 of the industrial truck 100, a control signal is output to the work device 30 as a control signal, which intervenes in the driving dynamics of the industrial truck 100. A risk to the occupational safety of the person 12 posed by the industrial truck 100 is thus reduced, and the performance of the work by the industrial truck 100 in the warehouse 2 is thus improved.

[0045] Reference symbol

[0046] warehouse

[0047] Environment

[0048] object

[0049] Human storage object

[0050] Environment detection sensors

[0051] Work equipment

[0052] Industrial truck

[0053] Control device, 114 interface

[0054] Import point cloud

[0055] Segmenting point cloud

[0056] Transfer point information

[0057] Determine distribution parameters

[0058] Classify object

[0059] Output control signal

Claims

Patent claims 1. A method for controlling a forklift truck (100) in a warehouse (2), comprising the step of reading (S1) a point cloud of the warehouse (2) in an environment (4) of the forklift truck (100), wherein the read point cloud contains point information of at least one object (10) located in the environment (4), characterized by the steps: converting (S3) the point information into a frequency distribution of the point information, determining (S4) a statistical distribution parameter of the point information in the frequency distribution, classifying (S5) the object (10) into a person (12) or into a warehouse equipment item (14) based on the statistical distribution parameter, and outputting (S6) a control signal for controlling the forklift truck (100) in the warehouse (2) based on a classification resulting from the classifying step (S5).

2. Method according to claim 1, characterized in that the imported point cloud contains point information from at least two objects (10) located in the environment (4), and wherein the method comprises, as a further step, segmenting (S2) the imported point cloud into at least one segment which contains point information from the point information of the imported point cloud assigned to one of the at least two objects (10), and wherein in the step of conversion (S3) the assigned point information is converted into the frequency distribution.

3. Method according to one of the preceding claims, characterized in that in the step of determining (S3) the statistical distribution parameter the distribution parameter is determined based on a dispersion parameter in the frequency distribution.

4. Method according to one of the preceding claims, characterized in that in the step of conversion (S3) the point information is converted into a histogram of the point information and in the step of determination (S4) of the statistical distribution parameter the distribution parameter is displayed in the histogram. is correct, whereby the distribution parameter is determined based on a dispersion parameter in the histogram.

5. Method according to one of the preceding claims, characterized in that the point information of the read-in point cloud has spatial point coordinates of the at least one object (10) located in the environment (4) and wherein in the step of conversion (S3) the spatial point coordinates are converted into the frequency distribution.

6. Method according to one of the preceding claims, characterized in that the point information of the read-in point cloud includes signal intensities of measurement signals reflected from the at least one object (10) located in the environment (4) for capturing the point cloud and wherein in the step of conversion (S3) the signal intensities are converted into the frequency distribution.

7. Method according to one of the preceding claims, characterized in that the point information of the read-in point cloud has spatial velocity coordinates of the at least one object (10) located in the environment (4) and wherein in the step of conversion (S3) the spatial velocity coordinates are converted into the frequency distribution.

8. Method according to one of the preceding claims, characterized in that in the step of outputting (S6) the control signal, a control signal to interrupt a work task performed by the industrial truck (100) is output to a work device (30) of the industrial truck (100) if the classification from the step of classifying (S4) results in the classification of the object (10) into a person (12).

9. Method according to one of the preceding claims, characterized in that in the step of outputting (S6) the control signal a control signal for limiting a driving dynamic of the industrial truck (100) is output if the classification from the step of classification (S4) results in a classification of the object (10) into a storage-related object (14).

10. Method according to one of the preceding claims, characterized in that the step of classifying (S4) the object (10) is carried out based on a machine learning model, wherein at least one of the frequency distribution and the statistical distribution parameter is read into the machine learning model.

11. Method according to claims 4 and 10, characterized in that the step of classifying (S4) the object (10) is carried out based on the machine learning model, wherein the histogram is read into the machine learning model.

12. Control device (110) for controlling a forklift truck (100) in a warehouse (2), with an interface (112) for reading a point cloud of the warehouse (2) in an environment (4) of the forklift truck (100), wherein the read point cloud has point information of at least one object (10) located in the environment (4), characterized in that the control device (110) is configured to convert the point information into a frequency distribution of the point information, to determine a statistical distribution parameter of the point information in the frequency distribution, to classify the object (10) in a classification into a person (12) or into a warehouse item (14) based on the statistical distribution parameter, and that the control device (110) has an interface (114) for outputting a control signal for controlling the forklift truck (100) in the warehouse (2) based on the resulting classification.

13. Industrial truck (100), with a control device (1 10) for controlling the industrial truck (100) according to claim 12.

Images