Method and control device for controlling a transport tool

Abstract

The invention relates to a method for controlling a handling tool in a warehouse, comprising the following steps: reading (S1) a point cloud of the warehouse in the surroundings of the handling tool, the read point cloud having point information of at least one object located in the surroundings; converting (S3) the point information into a probability distribution of the point information; determining (S4) statistical distribution parameters of the point information in the probability distribution; classifying (S5) the objects as personnel or storage technology objects based on the statistical distribution parameters; and issuing (S6) a control signal for controlling the handling tool in the warehouse on the basis of the classification result obtained from the classification (S5) step. The invention further relates to a control device for carrying out the method and to a handling tool having such a control device.

Description

Technical Field

[0001] The invention relates to a method and a control device for controlling a handling tool in a warehouse. The invention also relates to a handling tool having such a control device. Background Art

[0002] It is known from the prior art to control a vehicle based on detection of the surrounding environment. For example, it is known from DE 10 2018 117 428 A1 to detect objects in the vehicle's surroundings and to classify them based on their contours. In addition, it is known from WO 2020 / 049154 A1 to extract sensor-independent features in order to perform object classification based on a neural network. Summary of the invention

[0003] The present invention relates in one aspect to a method for controlling a handling tool in a warehouse. The handling tool may be a handling mechanism for transporting goods. The handling tool may be an industrial vehicle or a handling device. The handling tool is, for example, a forklift. The warehouse may have a warehouse management in production or logistics. Goods to be transported by the handling tool may be stored in the warehouse.

[0004] The method includes reading a point cloud of a warehouse in the surroundings of a handling tool as a step. The method may include detecting the point cloud using a surrounding environment detection sensor system as another step. The surrounding environment detection sensor system may be arranged on the handling tool to detect the point cloud. The method includes sending a control signal for detecting the point cloud using the surrounding environment detection sensor system as a further step.

[0005] The surrounding environment detection sensor system may have a distance sensor. Alternatively or additionally, the surrounding environment detection sensor system may have an image detection sensor. The distance sensor may be designed as a scanning sensor. The distance sensor may be, for example, a laser scanner, a radar measuring device, or an ultrasonic measuring device. The laser scanner, the radar measuring device, or the ultrasonic measuring device may be designed as a two-dimensional or three-dimensional scanning sensor.

[0006] If the surrounding environment detection sensor system has a distance measuring sensor, the warehouse can be detected point by point in the surrounding environment of the handling tool or scanned. The distance measuring sensor can be used to detect the point cloud of the warehouse in the surrounding environment of the handling tool. The point cloud can be a two-dimensional or three-dimensional point cloud. The image detection sensor can be designed as a camera system. The camera system can have an RGB camera. The camera system can include a stereo camera system. If the surrounding environment detection sensor system alternatively or in addition to the distance measuring sensor also has an image detection sensor, the image of the warehouse in the surrounding environment of the handling tool can be detected or photographed. The image detection sensor can be used to detect a point cloud with image points of the warehouse.

[0007] The read point cloud has point information of at least one object in the surrounding environment. The point information may have a measurement point detected by a distance measuring sensor. Alternatively or additionally, the point information may have an image point detected by an image detection sensor. The point information may have meta information about the measurement point. Alternatively or additionally, the point information may have meta information about the image point. The point information may have at least one point coordinate of the measurement point. Alternatively or additionally, the point information may have at least one color information in the visible or invisible spectral range of the image point.

[0008] The method comprises, as a further step, converting the point information into a probability distribution of the point information. The conversion of the point information may comprise converting the point information from a sensor-related coordinate system into a coordinate system of the probability distribution. The probability distribution may be a one-dimensional or multi-dimensional probability distribution. The probability distribution may be a statistical function describing the probability of the occurrence of the point information. Thus, a subset of the read point cloud may be converted into a probability distribution. Thus, in the step of converting the point information, the multi-dimensional point cloud may be converted into a one-dimensional probability distribution of the point information. To this end, the point information may form a one-dimensional subset of the point cloud. The point information may be derived therefrom as a one-dimensional subset of the point cloud. Thus, the subsequent steps of the method may be performed in a particularly efficient manner based on the converted probability distribution of the point information.

[0009] The method comprises determining a statistical distribution parameter of the point information in the probability distribution as a further step. The statistical distribution parameter may be a discreteness of the point information in the probability distribution. Thus, the statistical distribution parameter may be, for example, a standard deviation or a variance of the point information in the probability distribution. Thus, the inhomogeneity of the point information may be determined using the statistical distribution parameter.

[0010] The method comprises, as a further step, classifying the object as a person or a storage technology object based on a statistical distribution parameter. The object may be an obstructing object of a handling tool or a working object of the handling tool. The method may be used to control the handling tool to avoid collision with the obstructing object. The method may be used to control the handling tool to improve the movement of the handling tool towards the working object. According to one embodiment, the categories into which the objects are classified include persons and storage technology objects.

[0011] In the classification step, the object can be classified as a person or a storage technology object depending on the threshold value of the statistical distribution parameter. When the threshold value is exceeded, the object can be classified as a person. When the threshold value is below the threshold value, the object can be classified as a storage technology object. The classification of objects based on statistical distribution parameters is based on the recognition that point information related to people may be more uneven than point information related to storage technology objects. Due to the unevenness, a greater dispersion of the corresponding statistical distribution parameters in the probability distribution, such as point information, may occur.

[0012] The method comprises, as a further step, issuing a control signal for controlling a handling tool in a warehouse based on a classification result obtained from the classification step. According to one embodiment, the classification result obtained comprises classification as a person and classification as a storage technology object. The control signal can be issued depending on whether the object is classified as a person or a storage technology object.

[0013] Thus, with the present invention, it is possible to carry out a classification based on the statistical distribution of the point information as a person or as a storage technology object, so as to automatically control the handling means depending on the person or storage technology object identified by means of the classification. Depending on whether the point information is about a person or about a storage technology object, the statistical distribution parameters may be significantly different. Thus, the classification step and the step of controlling the handling means can be reliably carried out.

[0014] According to one embodiment of the method, the read point cloud may have point information of at least two objects in the surrounding environment. The method may include, as a further step, segmenting the read point cloud into at least one segment. The segment may have point information assigned to one of the at least two objects from the point information of the read point cloud. The segmentation may be performed based on the point information. Thus, a segment may have point information associated with only one object. In the conversion step, the assigned point information may be converted into a probability distribution. Thus, the probability distribution and the determined statistical distribution parameters may be associated with the segmented object, which may be a person or a storage technology object.

[0015] The storage technology object may include, for example, a shelf or a pallet or such an object. The storage technology object may also be an object of the infrastructure of the warehouse. The personnel may be pedestrians in the warehouse. The steps of the method may be performed for a plurality of objects in the surrounding environment. Thus, the method may be reliably performed even if the personnel and the storage technology object are in the surrounding environment of the handling tool.

[0016] According to another embodiment of the method, in the step of determining the statistical distribution parameters, the distribution parameters are determined based on discrete parameters in the probability distribution. The discrete parameters can be the standard deviation or variance of the point information in the probability distribution. Thus, the step of classifying the objects can be performed in dependence on the discrete parameters. If the discrete parameters exceed a predetermined threshold value of the discrete parameters, the object can be classified as a person. If the discrete parameters are below a predetermined threshold value, the object can be classified as a storage technology object. Thus, classification can be effectively performed based on the discreteness of the point information.

[0017] According to another embodiment of the method, in the conversion step, the point information can be converted into a histogram of the point information. The point information can be embodied as a mapping of the point information in the histogram. In the step of determining the statistical distribution parameters, the distribution parameters can be determined in the histogram or in a sub-area of ​​the histogram. The distribution parameters can be determined based on discrete parameters in the histogram. The statistical distribution parameters can be determined in the histogram functionally or based on the image. Therefore, the distribution parameters can be determined in a particularly effective manner in the mapping of the point information in the histogram using image processing methods. Therefore, the method can also be performed in an advantageous manner without using artificial intelligence (AI) measures.

[0018] According to another embodiment, the point information of the read point cloud has spatial point coordinates of at least one object in the surrounding environment. The spatial point coordinates may be at least one-dimensional point coordinates of at least one object in the surrounding environment. In the conversion step, the spatial point coordinates may be converted into a probability distribution. Thus, the determined statistical distribution parameters or discrete parameters may define the discreteness of the spatial point coordinates of the object. Thus, the described classification may be performed depending on the spatial extension of the object in at least one dimension.

[0019] According to another embodiment of the method, the point information of the read point cloud has a signal strength of a measurement signal reflected on at least one object in the surrounding environment for detecting the point cloud. In a conversion step, the signal strength can be converted into a probability distribution. Thus, the determined statistical distribution parameter or discrete parameter can define the discreteness of the signal strength of the measurement signal reflected on the object. Thus, the classification can be performed depending on the material properties or the geometry of the object that may influence the signal strength.

[0020] According to another embodiment of the method, the point information of the read point cloud may have the spatial velocity coordinates of at least one object in the surrounding environment. If the surrounding environment detection sensor system has a radar device, the spatial velocity coordinates can be determined from the measurement data of the radar device. In the conversion step, the spatial velocity coordinates can be converted into a probability distribution. The determined statistical distribution parameters or discrete parameters can define the discreteness of the spatial velocity coordinates. Thus, the classification can be performed depending on whether the areas of the object move relative to each other or whether the object is a rigid object.

[0021] According to another embodiment of the method, in the step of issuing a control signal, a control signal for interrupting the work task performed by the transport tool is issued to the work device of the transport tool. The work device may be a driving device for driving the transport tool. The work device may also be a transport device or a lifting device of the transport tool for transporting or lifting goods. When the classification result that the object is classified as a person is obtained from the classification step, a control signal for interrupting the work task may be issued. Therefore, the movement of the transport tool when performing the work task may be stopped based on the control signal to ensure the work safety of the person.

[0022] According to another embodiment of the method, in the step of emitting a control signal, a control signal for controlling or limiting the driving dynamics of the handling tool can be emitted. A control signal for limiting the longitudinal dynamics of the handling tool can be emitted. Thus, the speed or acceleration of the handling tool can be limited. A control signal for braking the handling tool can also be emitted. A control signal for limiting the lateral dynamics of the handling tool can be emitted. Thus, the steering angle of the handling tool can be limited. If the classification step results in that the object is classified as a storage technology object, a control signal for limiting the driving dynamics can be emitted. Alternatively, a control signal can also be emitted if the classification step results in that the object is classified as a person.

[0023] According to another embodiment of the method, the step of classifying the objects is performed based on a machine learning model. The machine learning model can have artificial intelligence measures or neural networks. According to this embodiment, at least one of the probability distribution and the statistical distribution parameter is read into the machine learning model. Thus, the probability distribution or the statistical distribution parameter can first form an input parameter for the machine learning model in an advantageous manner.

[0024] According to another embodiment of the method, the step of classifying the objects is performed based on a machine learning model, wherein the histogram is read into the machine learning model. Thus, the classification of the objects can be performed based on the histogram in a particularly efficient manner. In this case, an image-based evaluation of the histogram is particularly advantageous.

[0025] In another aspect, the present invention relates to a control device for controlling a handling tool in a warehouse. The control device has an interface for reading a point cloud of the warehouse in the surroundings of the handling tool. The read point cloud has point information of at least one object in the surroundings.

[0026] The control device is configured to convert the point information into a probability distribution of the point information. The control device is also configured to determine a statistical distribution parameter of the point information in the probability distribution. The control device is also configured to classify the object as a person or a storage technology object based on the statistical distribution parameter. The control device also has an interface for issuing a control signal for controlling a handling tool in the warehouse based on the obtained classification result.

[0027] The control device can be set up to perform the method according to the aforementioned aspect. The control device can be arranged on the handling tool. The features and embodiments of the method according to the aforementioned aspect can be corresponding features and embodiments of the control device, wherein the control device can be set up to perform each step described for the method. The control device can have corresponding units for performing the method steps.

[0028] In another aspect, the present invention relates to a handling tool. The handling tool can be configured as described in the above aspect. The handling tool has a control device according to the above aspect for controlling the handling tool. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 A transport tool and a control device for controlling the transport tool according to various embodiments of the present invention are shown.

[0030] Figure 2 A handling tool in a warehouse is schematically shown to illustrate the invention.

[0031] Figure 3 A flow chart showing method steps for executing a method for controlling a handling tool according to an embodiment of the present invention. Specific implementation plan

[0032] Figure 1 The figure shows a handling tool 100 having a working device 30 for carrying out a working task of the handling tool 100. A surroundings detection sensor system 20 is arranged on the handling tool 100, with which a point cloud in the surroundings 4 of the handling tool 100 is detected. The point cloud has preprocessed measurement data of the surroundings detection sensor system 20. The point cloud has points related to the surroundings 4 of the handling tool 100 and the surroundings 4 of the handling tool 100. Figure 2 10 related points of the object are shown.

[0033] The handling tool 100 has a control device 110 for controlling the working device 30. The control device 110 reads and processes the point cloud detected by the surrounding environment detection sensor system 20 via an interface 112 so as to control the working device 30 based on the processed point information. According to one embodiment, the point cloud has point information read by the control device 110. According to another embodiment, the point information of the read point cloud is derived from the point cloud by the control device 110. The control device 110 sends a control signal via an interface 114 to control the working device 30 based on the point information.

[0034] Figure 2A handling tool 100 is shown in a warehouse 2. In the warehouse 2, objects 10 are located in the surroundings 4 of the handling tool 100. The point cloud detected in the surroundings 4 has measurement data or points related to the objects 10. One of the objects 10 is a person 12 who is in the warehouse 2, while another of the objects 10 is a storage technology object 14 located in the warehouse 2, which according to one embodiment is a shelf.

[0035] Figure 3 Method steps S1 to S6 for controlling the handling tool 100 , which are performed by the control device 110 , are shown.

[0036] In a first step S1, the control device 110 performs a reading of a point cloud via the interface 112. The read point cloud has point information of an object 10 in the surroundings 4. According to one embodiment, the point information has point coordinates. According to another embodiment, the point information has a signal strength of a measurement signal of the surroundings detection sensor system 20 reflected on the object 10.

[0037] In the second step S2, the control device 110 performs segmentation of the point cloud or point information. The read point cloud is segmented into at least two sections. The first section has point information assigned to one of the objects 10. The second section has point information assigned to another of the objects 10. The segmentation step can be used to distinguish point information related to different objects 10. The subsequent steps of the method are performed for the respectively assigned point information.

[0038] In the third step S3, the control device 110 performs conversion of the point information into a probability distribution. The segmented point information is converted into a histogram having the probability of the segmented point information. According to a corresponding embodiment, the histogram has the probability of the point coordinates and the probability of the signal strength.

[0039] In a fourth step S4, the control device 110 performs a determination of distribution parameters in the probability distribution. Distribution parameters are discrete parameters of the probability of the histogram. The histogram forms an input parameter of a machine learning model for classifying the object 10 as a person 12 or a storage technology object 14 based on the discretization of the segmented point information. According to one embodiment, the discrete parameter constitutes a classification parameter in the machine learning model.

[0040] In the fifth step S5, the control device 110 performs classification of the object 10 as a person 12 or a storage technology object 14. Classification as a person 12 or a storage technology object 14 is an output variable of the machine learning model, which is based on a discrete parameter according to the described embodiment. The machine learning model is based on the assumption that the point coordinates or signal strengths assigned to the person 12 are more discrete than the point coordinates or signal strengths assigned to the storage technology object 14. The relatively discrete point coordinates of the storage technology object 14 are based on the assumption that the storage technology object 14 is an object 10 that is flatter or less curved than the person 12 and is at least related to one coordinate axis. The more discrete signal strength for the person 12 is based on the assumption that the person has an object surface that can be detected by the surrounding environment detection sensor system 20 compared to the storage technology object 14, and the object surface has more material and curvature, which leads to a more uneven and therefore more discrete signal strength of the measurement signal reflected on the object surface.

[0041] In a sixth step S6, the control device 110 executes the sending of a control signal to the working device 30 via the interface 114. If the classification result from the previous step S5 indicates that the person 12 is in the surroundings 4 of the handling tool 100, a control signal for interrupting the execution of the work task is sent to the working device 30 as a control signal. If the classification result from the previous step S5 indicates that the storage technology object 14 is in the surroundings 4 of the handling tool 100, a control signal for intervening in the driving dynamics of the handling tool 100 is sent to the working device 30 as a control signal. As a result, the dangers posed by the handling tool 100 to the person 12 in terms of their work safety can be reduced, and the execution of the work of the handling tool 100 in the warehouse 2 can be improved.

[0042] Reference numerals

[0043] 2 Warehouse

[0044] 4 Surrounding environment

[0045] 10 Objects

[0046] 12 people

[0047] 14 Storage technology objects

[0048] 20 Surrounding environment detection sensor system

[0049] 30 Operating equipment

[0050] 100 Handling tools

[0051] 110 Control device

[0052] 112, 114 interface

[0053] S1 Read point cloud

[0054] S2 Segmentation Point Cloud

[0055] S3 conversion point information

[0056] S4 Determine the distribution parameters

[0057] S5 Classify objects

[0058] S6 sends a control signal

Claims

1. A method for controlling a handling tool (100) in a warehouse (2), the method comprising the step of reading (S1) a point cloud of the warehouse (2) in the surroundings (4) of the handling tool (100), wherein: The read point cloud has point information of at least one object (10) in the surrounding environment (4), and is characterized by the following steps: converting (S3) the point information into a probability distribution of the point information; determining (S4) statistical distribution parameters of the point information in the probability distribution; classifying (S5) the object (10) as a person (12) or a warehouse technology object (14) based on the statistical distribution parameters, and issuing (S6) a control signal for controlling the handling tool (100) in the warehouse (2) based on the classification result obtained from the classification (S5) step.

2. The method according to claim 1, characterized in that The read point cloud has point information of at least two objects (10) located in the surrounding environment (4), and the method comprises segmenting (S2) the read point cloud into at least one segment, as a further step, the segment having point information assigned to one of the at least two objects (10) from the point information of the read point cloud, and wherein, in a conversion (S3) step, the assigned point information is converted into a probability distribution.

3. The method according to any one of the preceding claims, characterized in that In the step of determining (S3) the statistical distribution parameter, the distribution parameter is determined based on discrete parameters in the probability distribution.

4. The method according to any one of the preceding claims, characterized in that In the converting (S3) step, the point information is converted into a histogram of the point information, and in the determining (S4) step of the statistical distribution parameter, the distribution parameter is determined in the histogram, wherein the distribution parameter is determined based on a discrete parameter in the histogram.

5. The method according to any one of the preceding claims, characterized in that The point information of the read point cloud has spatial point coordinates of at least one object (10) in the surrounding environment (4), and in the conversion (S3) step, the spatial point coordinates are converted into the probability distribution.

6. The method according to any one of the preceding claims, characterized in that The point information of the read point cloud has a signal strength of a measurement signal reflected on at least one object (10) in the surroundings (4) for detecting the point cloud, and wherein, in a conversion (S3) step, the signal strength is converted into the probability distribution.

7. The method according to any one of the preceding claims, characterized in that The point information of the read point cloud has the spatial velocity coordinates of at least one object (10) in the surrounding environment (4), and in the conversion (S3) step, the spatial velocity coordinates are converted into the probability distribution.

8. The method according to any one of the preceding claims, characterized in that In the step of issuing (S6) the control signal, when the classification result that the object (10) is classified as a person (12) is obtained from the classification (S4) step, a control signal for interrupting the working task performed by the transport tool (100) is issued to the working device (30) of the transport tool (100).

9. The method according to any one of the preceding claims, characterized in that In the step of issuing (S6) the control signal, a control signal for limiting the driving dynamics of the handling tool (100) is issued if the classification result of the classification (S4) is that the object (10) is classified as a storage technology object (14).

10. The method according to any one of the preceding claims, characterized in that The step of classifying (S4) the object (10) is performed based on a machine learning model, wherein at least one of the probability distribution and the statistical distribution parameter is read into the machine learning model.

11. The method according to claims 4 and 10, characterized in that The step of classifying (S4) the object (10) is performed based on the machine learning model, wherein the histogram is read into the machine learning model.

12. A control device (110) for controlling a handling tool (100) in a warehouse (2), the control device having an interface (112) for reading a point cloud of the warehouse (2) in the surroundings (4) of the handling tool (100), wherein: The read point cloud has point information of at least one object (10) in the surrounding environment (4), characterized in that the control device (110) is configured to convert the point information into a probability distribution of the point information, determine a statistical distribution parameter of the point information in the probability distribution, classify the object (10) as a person (12) or a storage technology object (14) based on the statistical distribution parameter, and the control device (110) has an interface (114) for issuing a control signal for controlling the handling tool (100) in the warehouse (2) based on the obtained classification result.

13. A handling tool (100) comprising a control device (110) for controlling the handling tool (100) according to claim 12.

Images