SYSTEM FOR EXECUTING WORK PROCESSES IN A LOGISTICS ENVIRONMENT

Description

[0001] The present invention relates to a system for carrying out work processes in a logistics environment, comprising a load transport device and an operating device located remotely from the load transport device.

[0002] For reasons of increasing the efficiency of workflows in logistics environments, a clear trend towards increased automation and autonomy of equipment involved in such processes has been observed recently. In particular, so-called driverless transport systems are currently a dominant topic, enabling the execution of predefined processes and equipped with highly integrated sensors and actuators, as well as suitable control units and communication interfaces, for the most autonomous operation possible.However, it becomes apparent in this context that the complexity of the tasks that can be performed by such devices, or by other automated or self-contained systems, is limited in practice, since the automated execution of even more complex tasks would require an unacceptably or disproportionately large effort in equipping and programming the corresponding components. Examples of fully automated or fully self-contained devices of this kind are known, for instance, from DE 10 2015 220 091 A1 and US 10,147,069 B2.

[0003] Furthermore, reference is made to US 2013 / 190 963 A1 and US 2019 / 135 598 A1, which each disclose systems of the same type.

[0004] The present invention is therefore directed towards the problem of enabling the execution of essentially arbitrarily complex tasks and workflows in a logistics environment using a load handling device, which may be, for example, a forklift truck or a storage and retrieval machine, without requiring a disproportionately large effort for the complete automation of such processes. A simple example could be the handling of non-standardized containers or other irregularly shaped goods, i.e., goods whose state of matter changes, or, for example, spherical goods that are located or moved in an undefined manner in a storage compartment and are therefore difficult to pick independently.While it is conceivable to handle such goods automatically under certain circumstances, a multitude of possible conditions and operating modes must be considered when equipping and programming the corresponding system. Furthermore, numerous risks of operator error remain, as it is practically impossible to anticipate all eventualities. Moreover, many other examples exist where automated or autonomous systems in logistics facilities, driven by the pursuit of further automation of even complex processes, reach their economically viable limits.

[0005] The system according to the invention for carrying out work processes in a logistics environment comprises, for solving the problem outlined above, a load transport device, in turn comprising a central control unit, at least one communication unit, at least one optical camera, a plurality of actuators and / or working devices operable by the central control unit, and at least one sensor unit for detecting operating parameters of the load transport device and / or properties of the environment of the load transport device, which is operationally coupled to the central control unit, wherein the load transport device is configured to be operated at least temporarily in an autonomous mode by means of the control unit and the actuators on the basis of work instructions transmitted via the communication unit, as well as an operating device remote from the load transport device, comprising a communication unit,The device comprises a central control unit of the load transport device, which is in bidirectional data communication with the communication unit of the load transport device for the transmission of image and control data; a display unit, which is configured to display image data captured by the optical camera of the load transport device and transmitted via the communication unit; and an operating unit, which is configured to receive control commands from an operator, which are then transmitted to the load transport device via the communication units. According to the invention, the central control unit of the load transport device is further configured to switch the load transport device from autonomous mode to remote-controlled mode based on detections made by the sensor units and at least one predetermined criterion.in which the operation of the load transport device can be controlled by means of control commands entered at the remote control device.

[0006] Accordingly, a key aspect of the present invention is to provide a load transport device, initially operable in an autonomous mode, with all necessary components, which is also capable of recognizing predetermined states or conditions in which manual intervention by a remote human operator is necessary. The system according to the invention then allows this human operator to instruct the corresponding functions of the load transport device in a remote-controlled mode, and provides visual feedback via the optical camera of the load transport device and the display unit of the remote control. The corresponding detections by the at least one sensor unit can, for example, be performed periodically at a predefinable or adjustable frequency and compared with the at least one predetermined criterion.At the same time, the application should also include embodiments in which, for example, an external unit detects the state of the load transport device, such as its position at a predetermined location, which triggers a switch to remote-controlled mode. In such an example, the at least one sensor unit can even be physically located away from the load transport device, although for the sake of clarity and readability of this application, it is nevertheless considered and treated as part of it.

[0007] In the example of non-standardized containers already briefly mentioned above, according to the invention, when such a container is detected and with knowledge of the work instruction to pick the corresponding container, the device switches from autonomous mode to remote-controlled mode and a remote operator can control a corresponding work device on the load transport device in a suitable way for the picking process by means of control commands entered via the operating unit.

[0008] For the two interconnected communication units, known protocols and standards can be used; for example, technologies such as IEEE-802.11, TCP / IP and 4G / 5G or proprietary standards can be used to transmit the camera image and control commands. Of course, further data can also be transmitted, such as telemetry and / or sensor data from the load transport device or other components of the logistics system, which can also be displayed to the human operator in a suitable manner to support him in remotely controlling the load transport device.

[0009] For this purpose, the display unit of the remote control device can be equipped according to the data and image formats to be reproduced and can be formed, for example, by the monitor of an end device, such as a personal computer, or even VR glasses.

[0010] Regarding the types of sensor units that can be used in the load handling device, there are initially no restrictions, and a wide variety of sensor types can be used, such as position sensors, orientation sensors, sensors for monitoring work functions, optical sensors, acoustic sensors, and distance sensors. Furthermore, the load handling device and / or the remote control unit can receive and process additional sensor data or image data from sensors or cameras that are assigned to external components of the logistics system and are therefore static relative to the moving load handling device. This allows for the simplification, safety, or similar purposes of the autonomous operation and / or remote control of the load handling device by the human operator using additional external data.

[0011] Similarly, the actuators and / or working devices operable by the central control unit can also be selected from a variety of known types of such components, which are already used, for example, in fully autonomous or self-sufficient driverless transport systems known from the prior art. These may include, for example, a drive and steering system for the load transport device, a manipulator, a lifting frame, a side-shift device, or individual actuators forming part of these components. The selection and number of such controllable components therefore only need to be tailored to the specific applications and scenarios intended for the system according to the invention.

[0012] The same applies to the control unit of the remote control device, which may include, for example, a joystick, a steering wheel, a data glove and / or a touchscreen, so that different input devices are conceivable for different actuators and / or work equipment intended for remote control.

[0013] In an advantageous embodiment, the optical camera of the load transport device can also be aligned and / or adjusted by means of the control unit, so that the human operator entrusted with the remote control can adjust the field of view displayed on his display unit within predefined parameters and, for example, zoom in or out to recognize details or to obtain a better overview of the currently recorded scene.

[0014] Furthermore, a warehouse management system can be assigned to the system according to the invention. This system, in turn, is connected to the communication unit of the load transport device via a communication unit and is configured to send work instructions to the load transport device. Here, too, existing technologies can be used, since the warehouse management system is not fundamentally responsible for the switching between autonomous and remote-controlled operation according to the invention. Nevertheless, the technology described above allows for the execution of more complex processes by involving a human operator, thereby enabling workflows and processes not previously possible with warehouse management systems.Accordingly, corresponding work instructions can be relatively simple, for example containing only an instruction for a load transport device to move to a specific position within a logistics facility, or complex tasks such as picking up goods at a predetermined first point, possibly identifying the goods from a plurality of goods present at the corresponding position, and subsequently transporting and transferring the specified goods to a position to be reached by an intermediate journey, whereby the transition to remote-controlled mode is then decided on the vehicle side according to the aspects described above.

[0015] Furthermore, the system according to the invention offers the possibility of combining several load transport devices and a single remote operating device, as well as optionally the aforementioned single warehouse management system, wherein the load transport devices can be operated individually and selectively in remote-controlled mode. Thus, the efficiency of the system according to the invention is improved in that a single human operator can operate several load transport devices in corresponding remote-controlled modes, so that a one-to-one mapping between load transport device and remote operator is not necessary.Of course, several remote operating devices could also be provided for a larger number of load transport devices, whereby in any case the processing of work processes to be remotely controlled by the human operator(s) can be prioritized and / or distributed via suitable algorithms, for example, work steps with a higher urgency can be given a higher priority and thus be processed at an earlier time or according to a strict first-in-first-out batch processing principle.

[0016] It should also be noted that the switch from remote operation to autonomous operating mode can take place in various ways, for example by a corresponding input from the operator when he believes that the remotely controlled activity to be carried out by him is completed, or also by the detection of such a state by one of the sensor units.

[0017] Further features and advantages of the present invention will become even clearer from the following description of an exemplary embodiment when considered together with the accompanying figures. These show in detail: Fig. 1 a schematic representation of the components of a system according to the invention; Fig. 2 a flowchart of a method for operating the system. Figure 1 ; and Fig. 3 a schematic representation of an embodiment of a load transport device and a remote control device from the system made up of Fig. 1 .

[0018] In Figure 1A schematic representation of a system 10 according to the invention is shown first, comprising a forklift truck 20 acting as a load transport device within the meaning of the present invention, an operating device 30 located remotely from the forklift truck 20, and a warehouse management system 40. It should be noted that the load transport device could alternatively also be formed, for example, by a storage and retrieval machine or similar device.

[0019] The industrial truck 20 comprises a central control unit 202, which handles all data processing tasks within the industrial truck 20 and can be either monolithic or distributed across several subunits, for example, a plurality of microchips. The central control unit 202 also includes at least one memory element 202a, in which functional software for operating the individual functions of the control unit 202 is stored and in which further specific data of a logistics facility can be stored, such as topology data and the like. Furthermore, data relating to the predetermined criteria for transitioning the industrial truck 20 from autonomous to remote-controlled mode is stored in the memory unit 202 within the framework of the method according to the invention.

[0020] Furthermore, the industrial truck 20 includes at least one communication unit 204, which can be configured to use communication protocols of both public and proprietary nature that are known per se. For example, protocols commonly used for transmitting large amounts of data, such as IEEE-802.11 or 4G / 5G, can be used.

[0021] One of the data sources whose data is to be transmitted via the communication unit 204 is an optical camera 206, which is arranged at a suitable position on the industrial truck 20 for carrying out the method according to the invention. This optical camera 206 can also be coupled to the industrial truck 20 in a remotely controllable manner, so that the exact position and viewing angle of the optical camera 206 can be adjusted within predefined limits by the operator of the remote control unit 30 described below.

[0022] Furthermore, the industrial truck 20 is equipped with a plurality of actuators and / or work devices 210, which can be operated by the central control unit 202 and perform typical tasks during the operation of the industrial truck 20, such as its drive and steering, or tasks related to the handling of objects. These may include, for example, manipulators or components of lifting masts, side-shift frames, or similar devices. Finally, the industrial truck 20 also includes at least one sensor unit 212 for recording operating parameters of the industrial truck 20 and / or properties of the industrial truck 20's environment, which are also operationally coupled to the central control unit 202.In this case, the industrial truck 20 is equipped to be operated autonomously, at least temporarily, by means of the control unit 202 and the actuators 210 controlled by it, based on the data supplied by the sensor units 212, in order to execute work instructions transmitted via the communication unit 204, which may in particular originate from the warehouse management system 40.

[0023] The remote control unit 30 also comprises a control unit 302, which processes at least the data generated within the scope of the method according to the invention, wherein the control unit 30 can be formed, for example, by a conventional PC with extended functionality. In particular, the remote control unit 30 also comprises a corresponding communication unit 304, with which a bidirectional data connection can be established with the communication unit 204 of the industrial truck 20, via which, on the one hand, data from the optical camera 206 for display on a screen 306 of the remote control unit 30 can be communicated in one direction, and, on the other hand, control commands entered by an operator via a control unit 308 of the remote control unit 30 can be communicated.

[0024] In addition to the images already mentioned, captured by camera 206, further data can also be displayed on the display unit 306, for example, processed sensor data in a manner suitable for a human operator, which has been pre-processed for this purpose by the control unit 302 of the remote operating device 30. This could, for example, be a schematic map of a logistics facility traversed by the industrial truck 20, in which the position of the industrial truck 20 and, if applicable, other traffic participants equipped with transponders are schematically represented.

[0025] Furthermore, reference should be made to the warehouse management system 40, by means of which data relating to the aforementioned work instructions can be transmitted to the industrial truck 20 via a further communication unit 402, and which may optionally be configured for further data transmission to the remote control unit 30 for greater integration of the system 10. A further industrial truck 20' is also indicated, albeit in simplified terms, which can likewise be remotely controlled by means of the remote control unit 30 and can exchange data with the warehouse management system 40. By enabling the control of several industrial trucks 20, 20' via a single warehouse management system 40 and a single remote control unit 30, further advantageous synergies can be achieved in the system 10 according to the invention.

[0026] A method used solely for illustrative purposes and not claimed here, which is related to system 10 from Figure 1 furthermore, the process of carrying out this will now be demonstrated using the flowchart from Figure 2 The following explains the process. First, in step S0, the warehouse management system 40 sends a work instruction to the industrial truck 20, which it then executes autonomously in step S1. Meanwhile, in step S2, at least one operating parameter of the industrial truck 20 or a property of the industrial truck 20's environment is recorded by the at least one sensor unit 212. This recording can, for example, take place in a clocked manner with a predetermined or adjustable frequency.

[0027] In step S3, the data thus acquired by the at least one sensor unit 212 are compared with at least one predetermined criterion; if this criterion is not met (N), the process returns to step S2 until the next acquisition, while if the corresponding criterion (Y) is met by the sensor data, the process proceeds to step S4, in which the industrial truck 20 is switched to a remote-controlled mode.

[0028] The operator of the remote control unit 30 is informed of this switching operation in a suitable manner on the display unit 306, so that he can take over the operation of at least one actuator and / or work device 210 of the industrial truck 20 by means of the control unit 308, whereby the corresponding control commands are transmitted as already mentioned via the two communication units 304 and 204 and processed by the central control unit 202 of the industrial truck 20 to control the corresponding actuator or work device 210.

[0029] After the corresponding operation from step S5 has been completed by the human operator, they can, in step S6, instruct the industrial truck 20 to return to autonomous operation. In other variations of this process, this can also be automated based on data from sensors 212 and appropriate processing by the central control unit 202 of the industrial truck, once it is detected that the corresponding manually remote-controlled operation has been completed. The industrial truck 20 then returns to autonomous operation, allowing the process to begin again at step S1.

[0030] Lastly, in Figure 3 purely schematically, a possible embodiment of the industrial truck 20 and the remote control unit 30 of the system 10. Figure 1The diagram shows the industrial truck 20 as a vertical order picker, typically used in high-bay warehouses. In addition to the two cameras 206, which can also be configured as 3D cameras or 3D scanners, the diagram also shows the communication unit 204 and a work device configured as a manipulator 210, which are appropriately assigned to the vehicle body 22 and the lifting mast 24 of the industrial truck 20, respectively.

[0031] On the other hand, in the present embodiment, the remote control unit 30 is designed such that it comprises a display unit 306 in the form of a conventional computer monitor and a separate communication unit 304 that is coordinated with the communication unit 204 of the industrial truck 20, wherein the images from the two cameras 206 can be displayed either individually or simultaneously according to the operator's instructions for the remote control unit 30. Alternatively, the display unit could, for example, also be designed as 3D data glasses.

[0032] In contrast, the control unit 308 in the present embodiment is formed by a data glove, by means of which the manipulator 210 of the industrial truck 20 can be remotely controlled in order to be able to pick up even irregularly shaped goods or objects comfortably.

[0033] It should be noted at this point that the in Figure 3The embodiment shown is merely an illustrative example and there are numerous possibilities for modification or addition with regard to the design of the load transport device, the actuator 210 and the control unit 308.

Claims

1. System (10) for performing operations in a logistics environment, comprising: - a load transport device (20), comprising: o a central control unit (202); o at least one communication unit (204); ∘ at least one optical camera (206); ∘ a plurality of actuators and / or working devices (210) operable by the central control unit (202); and ∘ at least one sensor unit (212) for detecting operating parameters of the load transport device (20) and / or properties of the environment of the load transport device (20), which is operatively coupled to the central control unit (202); wherein the load transport device (20) is configured to be operated at least temporarily in an autonomous mode by means of the central control unit (202) and the actuators based on work instructions transmitted via the communication unit (204); and - an operating device (30) remote from the load transport device (20), comprising: ∘ a communication unit (304) that is in bidirectional data communication with the communication unit (204) of the load transport device (20) for transmitting image and control data; ∘ a display unit (306) that is configured to display image data recorded by the at least one optical camera (206) of the load transport device (20) and transmitted by means of the communication units (204, 304); and ∘ an operating unit (308) that is configured to receive control commands from an operator, which are then transmitted to the load transport device (20) by means of the communication units (204, 304); characterised in that the central control unit (202) of the load transport device (20) is further configured to transfer the load transport device (20) from the autonomous mode to a remote-controlled mode based on recordings made by the at least one sensor unit (212) and at least one predetermined criterion, in which remote-controlled mode the operation of the load transport device (20) can be controlled using control commands input at the remote operating device (30), wherein the transfer from the autonomous mode to the remote-controlled mode is carried out upon detection of a non-standardized container and with knowledge of a work instruction to pick the corresponding container, so that the operator can control one of the working devices of the load transport device (20) for a picking process by means of control commands input via the operating unit (308).

2. System (10) according to Claim 1, wherein the at least one sensor unit (212) of the load transport device (20) comprises at least one of the following types: position sensors, location sensors, sensors for monitoring work functions, optical sensors, acoustic sensors, and distance sensors.

3. System (10) according to any one of Claims 1 and 2, wherein the actuators and / or working devices (210) operable by the central control unit (202) comprise at least one of: a drive and steering system of the load transport device, a manipulator, a lifting frame, and a sideshift device.

4. System (10) according to any one of the preceding claims, wherein the operating unit (308) comprises at least one of: a joystick, a steering wheel, a data glove, and a touchscreen.

5. System (10) according to any one of the preceding claims, wherein the at least one optical camera (206) of the load transport device (20) can be aligned and / or adjusted by means of the operating unit.

6. System (10) according to any one of the preceding claims, further comprising a warehouse management system (40) which is in data communication with the communication unit (204) of the load transport device (20) via a communication unit (402) and is configured to send work instructions to the load transport device (20).

7. System (10) according to any one of the preceding claims, comprising a plurality of load transport devices (20, 20') and a single remote operating device (30) and optionally a single warehouse management system (40), wherein the load transport devices (20, 20') are individually selectively operable in the remote-controlled mode.