METHOD FOR COMPUTER-IMPLEMENTED CONFIGURATION OF A CONTROLLED DRIVE APPLICATION OF A LOGISTICS SYSTEM

Description

[0001] The present invention relates to a method and a device for the computer-implemented configuration of a controlled drive application of a logistics system.

[0002] A so-called "dynamic gapper" is a controlled drive application used in intralogistics. In this application, packages, which are usually conveyed in one direction by two or more parallel conveyor lines, are to be consolidated onto a single outbound conveyor line. The packages, which are located at undefined intervals on the two or more parallel conveyor lines, are to be sorted onto the outbound conveyor line using a consolidating unit and simultaneously spaced at defined intervals.

[0003] The bundles are combined and defined spacing is achieved using several conveyor belts on separate conveyor lines. The conveyor belts are driven by geared motors and controlled by a frequency converter. The setpoints for the speeds of each conveyor line are specified by a controller. The positions of the bundles on the conveyor lines are detected by sensors and processed in the controller. The position between the bundles is usually influenced by PI controllers that operate between each pair of bundles. Based on this information, the controller accelerates and decelerates the conveyor lines differently.

[0004] Due to the large number of parallel conveyor lines and several sub-conveyor belts, each of which must be accelerated and decelerated independently of one another, the control system is complex. Optimization for different package sizes and properties is complex. Adapting the control process to different mechanical configurations is a major challenge. For example, changing the number of conveyor lines or the length of the individual sub-conveyor lines has a significant impact on the control process and requires adjustments to the control system.

[0005] A fundamental overarching goal of the application is to be able to dynamically optimize the throughput of such a controlled drive application during the running process, while at the same time avoiding malfunctions of the controlled drive application, e.g. due to collisions of individual packages.

[0006] Gary Gang Jing et al "Modeling a controlled conveyor network with merging configuration" and Michael Johnstone et al "Simulationbased baggage handling system merge analysis" disclose a drive application.

[0007] It is an object of the invention to provide a method and a device for the computer-implemented configuration of a controlled drive application of a logistics system, which solves these problems in a simple and reliable manner.

[0008] These objects are achieved by a method according to the features of patent claim 1, a device according to the features of patent claim 9, a logistics system according to the features of claim 11 and a computer program according to the features of claim 12. Advantageous embodiments emerge from the dependent claims.

[0009] The invention proposes a method for the computer-implemented configuration of a controlled drive application of a logistics system. The logistics system comprises one or more parallel conveyor lines for piece goods. Piece goods are, in particular, packages that are to be fed, for example, to a sorting system. However, the present method is also suitable for all other types of piece goods. The conveyor lines each open into a converging unit in one conveying direction. Each of the conveyor lines consists of a plurality of sub-conveyor lines arranged one behind the other so that a piece of piece goods can be transferred without interruption from one sub-conveyor line to another, adjacent sub-conveyor line. The sub-conveyor lines are accelerated or decelerated by a respective assigned drive under the control of a computing unit. The acceleration or deceleration occurs individually for each sub-conveyor line.The acceleration or deceleration can also include an acceleration or deceleration value of 0. By selectively accelerating or decelerating a respective partial conveyor section, the combining unit located at the end of the conveyor sections is enabled to combine the piece goods onto a single output conveyor section at a defined distance. In other words, the piece goods are moved onto the output conveyor section at defined distances from one another.

[0010] According to the method of the present invention, the following steps are performed: A system model of the logistics system is determined based on operational data of the logistics system. The operational data is available for a plurality of points in time during the operation of the logistics system and includes, for each point in time, measured values ​​from sensors of the logistics system and changes in manipulated variables. The system model represents a simulation model that reflects the dynamics of the logistics system.

[0011] In a next step, a control function of the logistics system is determined, wherein the control function comprises at least configuration data for the drives. The control function is determined based on the system model by executing at least one control function in the system model, specifying one or more performance characteristics to be achieved, in which the operating data is simulated for a plurality of time steps. A reward measure is determined for each time step, wherein the control function is used as a control function in which a predefined fitness function, which aggregates the reward measures of a plurality of time steps, satisfies a predetermined criterion and, in particular, is maximized.

[0012] The method of the invention represents a simple method for the computer-implemented determination of configuration data for the drives assigned to the partial conveyor lines. Configuration data within the meaning of the invention are, in particular, control signals for the drives in order to accelerate or decelerate the assigned partial conveyor lines in a suitable manner in order to enable the combining unit to combine the piece goods fed via the several parallel conveyor lines onto a single output conveyor line with a defined distance.

[0013] In particular, the system model is determined using supervised learning methods. The use of a neural network or a recurrent neural network is particularly preferred. The control function is determined, in particular, using reinforcement learning.

[0014] By using machine learning methods, the effort required for manual adjustment of the control function is eliminated. The configuration data can thus be determined automatically. Using reinforcement learning methods to determine the control function enables the discovery of better and more reliable solutions than was previously possible through manual adjustment. This allows for greater throughput at the output side of the combining unit. Furthermore, it is possible to convey consecutive piece goods with greater uniformity along the output conveyor line.

[0015] The use of supervised learning methods to create the system model, combined with rapid control optimization, enables short-term adjustment of specified optimization goals in the form of performance characteristics to be achieved. This results in lower costs for commissioning the logistics system. Furthermore, the performance of the logistics system can be increased.

[0016] A practical design of the method involves processing the sum of the performance characteristic(s) over the simulated time steps as the fitness function. For example, the sum of a specific performance characteristic over time can be used as a fitness function, e.g., the average throughput of piece goods at the output of the consolidating unit.

[0017] One or more of the following parameters can be processed as a performance characteristic: an average throughput of piece goods at the output of the combining unit; a distance, in particular a minimum distance, between two piece goods conveyed directly one after the other (the distance between two piece goods conveyed directly one after the other is also referred to as the gap distance); the detection of a collision in the combining unit, in particular at its output; a distance uniformity measure that characterizes a deviation of the distances from an equidistance between two piece goods conveyed directly one after the other; a running speed of the partial conveyor sections of a respective conveyor section or of the conveyor sections as a whole. As explained above, the control can be implemented in the system model by specifying one or more performance characteristics to be achieved.

[0018] According to a further preferred embodiment, one or more input variables of the system model are varied to determine the control function. One or more of the following parameters can be processed as input variables of the system model: a respective speed of the partial conveyor sections (corresponding to a respective speed of the drives); a position of a respective piece of goods on a respective conveyor section or a partial conveyor section of a respective conveyor section; a size, in particular length, of a respective piece of goods; a throughput of piece goods at the exit of the combining unit; a collision in the combining unit, in particular at its exit. A collision can be detected, for example, by no more piece goods being detected after the exit of the combining unit after a predetermined period of time has elapsed.

[0019] The input variables that are varied to determine the control function of the system model preferably simultaneously represent the operating parameters of the system model. The parameters mentioned are thus available to determine the system model as operating data for a large number of times during the operation of the logistics system and include measured values ​​from sensors of the logistics system as well as changes in the manipulated variables, such as a change in the respective speed of the partial conveyor lines.

[0020] In addition to the method described above, the invention relates to a device for the computer-implemented configuration of a controlled drive application of a logistics system, wherein the device comprises a computing unit for controlling a drive assigned to each partial conveyor section for acceleration or deceleration, wherein the computing unit is designed to carry out the method according to one or more preferred embodiments of the invention.

[0021] The invention further relates to a logistics system with one or more parallel conveyor lines for piece goods, as described herein. According to the invention, the logistics system comprises a device according to one or more preferred embodiments of the type described herein.

[0022] Finally, the invention relates to a computer program with program code stored on a non-volatile, machine-readable carrier for carrying out a method according to one or more preferred embodiments of the invention when the program code is executed on a computer.

[0023] The invention is described in more detail below using an exemplary embodiment in the drawing. In the drawings: Fig. 1 shows a schematic representation of a logistics system with a computing unit for carrying out the method according to the invention for the computer-implemented configuration of the controlled drive application implemented therein; and Fig. 2 shows a schematic representation of the method steps carried out by the computing unit.

[0024] Fig. 1 shows a schematic representation of a logistics system 1 with a controlled drive application. The logistics system 1 comprises, for example, three conveyor lines 10, 20, 30 running parallel to one another, on each of which piece goods, in particular packages, can be conveyed in a conveying direction FR running from right to left. Each of the conveyor lines 10, 20, 30, which are of equal length and are also referred to as fingers, comprises a plurality of partial conveyor lines 11-13, 21-23, 31-33. The number of partial conveyor lines per conveyor line 10, 20, 30 is the same in the present exemplary embodiment, although this is not mandatory. The partial conveyor lines 11-13, 21-23, 31-33 of a respective conveyor line 10, 20, 30 can be of the same length or different lengths.

[0025] Each of the partial conveyor lines 11-13, 21-23, and 31-33 is assigned a drive 11A-13A, 21A-23A, and 31A-33A. By controlling the drives 11A-13A, 21A-23A, and 31A-33A accordingly using a processing unit, the partial conveyor lines 11-13, 21-23, and 31-33 can be individually accelerated or decelerated.

[0026] At the end of the conveyor lines 10, 20, 30, i.e., in the conveying direction FR, a combining unit 40 is arranged, to which the last partial conveyor lines 13, 23, 33 in the conveying direction FR transfer the piece goods they transport. A single output conveyor line 50 is arranged at an exit 41 of the combining unit 40. This can consist of one or more partial conveyor lines 51. The partial conveyor line(s) 51 are in turn driven by a drive 51A under the control of the computing unit 60.

[0027] The acceleration and deceleration of respective partial conveyor sections by means of suitable control signals for the drives 11A-13A, 21A-23A, 31A-33A makes it possible to transport piece goods transported on the parallel conveyor sections 10, 20, 30 to the combining unit 40 at staggered times. The combining unit 40 is thus able to convey the piece goods to the output conveyor section 50 in such a way that two consecutive piece goods are at a predetermined, defined distance from each other.

[0028] To enable the computing unit 60 to output suitable control signals for accelerating and decelerating the drives 11A-13A, 21A-23A, 31A-33A, a respective partial conveyor section 11-13, 21-23, 31-33 is provided with a number of respective sensors 11S-13S, 21S-23S, 31S-33S. The sensors 11S-13S, 21S-23S, 31S-33S comprise, in particular, light barriers for determining a respective transport speed of a piece of goods, a length of the piece of goods, a position, and / or a deviation from an expected position. The sensors optionally include, for example, speed sensors for detecting the speed of the drives 11A-13A, 21A-23A, 31A-33A, current sensors for detecting the motor currents of the drives 11A-13A, 21A-23A, 31A-33A, etc.

[0029] The supply of piece goods to the conveyor lines 10, 20, 30 takes place via respective transfer units 18, 28, 38, which are also designed, for example, as partial conveyor lines. The transfer units 18, 28, 38 also have a corresponding drive (not explicitly shown here) and a number of corresponding sensors 18S, 28S, 38S. The transfer units can be independent segments of the actual conveyor lines 10, 20, 30. However, the transfer units 18, 28, 38 can also represent a respective partial conveyor line of the assigned conveyor line 10, 20, 30.

[0030] For simplicity, Fig. 1 Only the transfer units 18, 28, 38 are equipped with corresponding sensors 18S, 28S, 38S. Corresponding measurement signals are fed to the computing unit 60 for further processing. A measurement signal is represented by a dotted line. For the sake of simplicity, not all measurement signals or signal lines required for transmission are shown.

[0031] The drives 11A-13A, 21A-23A, 31A-33A, and 51A assigned to the conveyor sections 11-13, 21-23, and 31-33 are controlled by dashed lines with corresponding control signals. For the sake of simplicity, not all control signals or control lines required for transmission are shown.

[0032] The method described below for the computer-implemented configuration of the controlled drive application of the logistics system 1 is carried out by the processing unit 60. However, the steps can also be carried out on a processing unit independent of the ultimate control of the logistics system 1. The procedure is shown schematically in Fig. 2 shown.

[0033] In a first step S1, a system model of the logistics system 1 is determined based on operating data BD of the logistics system. The operating data BD is available for a large number of times during the operation of the logistics system 1 and includes, for each time, measured values ​​from the sensors 11S-13S, 21S-23S, 31S-33S, 18S-38S, such as light barrier signals, motor currents, positions of the piece goods on the respective partial conveyor lines 11-13, 21-23, 31-33, 18-38, speeds of the drives 11A-13A, 21A-23A, 31A-33A, and speeds of the partial conveyor lines 11-13, 21-23, 31-33. In principle, not only operating data BD of the currently considered logistics system 1, but also operating data BD from other, preferably similar logistics systems, can be processed.

[0034] In addition, in step S1, changes in the manipulated variables, which include, for example, changes in speed or rotational speed of the drives 11A-13A, 21A-23A, 31A-33A, 18A-38A, are determined and processed for each point in time.

[0035] The system model is determined using supervised learning methods, particularly a neural network or a recurrent neural network. Since the procedure is well known, a detailed description will be omitted here.

[0036] In a second step S2, a control function of the logistics system 1 is determined. The control function REGF comprises at least configuration data KD for the drives 11A-13A, 21A-23A, 31A-33A, ie motor currents and / or speeds and the like, so that the associated partial conveyor sections 11-13, 21-23, 31-33 can be accelerated or decelerated in a suitable manner.

[0037] The control function REGF is determined based on the system model determined in step S1 by executing at least one control in the system model, specifying one or more performance characteristics to be achieved. As a performance characteristic, one or more of the following parameters can be processed, for example: an average throughput of piece goods at the output 41 of the combining unit 40; a, in particular minimum, distance between two piece goods conveyed directly one after the other, i.e. a gap distance; the detection of a collision in the combining unit 40, in particular at its output 41; a distance uniformity measure that characterizes a deviation of the distances from an equidistance between two piece goods conveyed directly one after the other, i.e.a uniformity of the gap spacing; as well as a running speed of the three conveyor lines of a particular conveyor line or of the conveyor lines as a whole, in order to achieve, for example, wear optimization.

[0038] During control, the operating data BD is simulated for a large number of time steps. A reward measure is determined for each time step. Finally, the control function REGF is used, in which a given fitness function, which aggregates the reward measures from a large number of time steps, meets a given criterion. In particular, the goal is to maximize the reward measures.

[0039] The control function REGF is determined using methods that are suitable for creating a control that increases the fitness function. The fitness function can, for example, be the sum of the performance characteristic(s) over the time steps, e.g. the average throughput. Such an approach can be achieved using reinforcement learning methods, where the fitness function is then processed, for example, the discounted sum of the expected rewards. This can be, for example, the throughput or the proximity to a desired gap spacing. Other objectives can be a narrow width of the distribution of the package spacings or a high running speed of the partial conveyor sections while maintaining certain distance quantiles of the piece goods. Since the individual objectives sometimes contradict one another, these must be balanced out using the learned actions of the control function REGF.

[0040] In the second step S2, an optimal control function REGF for the given fitness function can be learned through reinforcement learning based on interactions with the system model. The control function REGF is thus optimized by training against the system model of Logistics System 1. From this, the interpretable control function REGF can then be derived, which enables the generation of configuration data for drives 11A-13A, 21A-23A, and 31A-33A.

[0041] Using model-based reinforcement learning approaches, the control function REGF can be executed in the simulation of the system model, and a large number of time steps can be simulated. By varying typical model input variables, such as the size of the piece goods, their mass, friction coefficients, and the like, practical distributions can be generated using previously available operating data BD. This enables a high degree of robustness of the derived control function REGF and configuration data KD. Since the fitness function is simulated for each time step, the development of the reward measures can be approximately calculated.

[0042] The use of an interpretable representation of the control system allows for learning interpretable controls. These allow for the formalization of empirical knowledge and support on-site adaptations during the commissioning of the logistics system.

[0043] By combining a learned system model with fast optimization methods, such as particle swarm optimization, control systems can be quickly developed for changing optimization objectives. Multi-step optimization, similar to the well-known model predictive control, allows adaptation to changing optimization objectives without delay.

[0044] To determine optimal configuration data KD, operational data from other logistics systems can be used to learn a control function REGF using Q-function-based reinforcement learning methods. Alternatively, the system model can be trained with validity for all logistics systems, which then enables optimization for the logistics system under consideration.

Claims

1. Method for computer-implemented configuration of a controlled drive application of a logistics system (1), wherein the logistics system (1) comprises a plurality of parallel-running conveyor sections (10, 20, 30) for piece goods that each lead to a combining unit (40) in the conveying direction (FR), each of the conveyor sections (10, 20, 30) consisting of a plurality of conveyor subsections (11-13, 21-23, 31-33) that are accelerated or decelerated by a respectively associated drive (11A-13A, 21A-23A, 31A-33A) under the control of a computing unit (60) in order to render the combining unit (40) able to combine the piece goods onto a single output conveyor section (50) at defined intervals, comprising the steps of: - determining a system model of the logistics system (1) on the basis of operating data of the logistics system (1), the operating data (BD) being available for a multiplicity of times in the operation of the logistics system and comprising, for each time, measured values from sensors (18S-28S) of the logistics system and manipulated variable changes; - determining a control function of the logistics system (1), which control function comprises at least configuration data (KD) for the drives (11A-13A, 21A-23A, 31A-33A), on the basis of the system model by specifying one or more performance features to be attained in order to perform at least one control operation in the system model, involving simulating the operating data (BD) for a multiplicity of time steps, wherein a reward quantity is ascertained for each time step, wherein the control operation is used as the control function, involving a predefined fitness function that aggregates the reward quantities for a multiplicity of time steps satisfying a predetermined criterion.

2. Method according to Claim 1, in which the system model is determined using supervised learning methods.

3. Method according to Claim 1 or 2, in which the control function is determined by way of reinforcement learning.

4. Method according to one of the preceding claims, in which the sum of the performance feature (s) for the simulated time steps is processed as the fitness function.

5. Method according to one of the preceding claims, in which one or more of the following parameters are processed as a performance feature: - a mean throughput of piece goods at the output (41) of the combining unit (40); - an interval between two piece goods conveyed in direct succession; - the detection of a collision in the combining unit (40); - an interval uniformity measure that characterizes a deviation of the intervals from an equidistance between each pair of piece goods conveyed in direct succession; - a running speed of the conveyor subsections (11-13, 21-23, 31-33) of a respective conveyor section (10, 20, 30) or of all of the conveyor sections (10, 20, 30).

6. Method according to one of the preceding claims, in which the control function is determined by varying one or more input variables of the system model.

7. Method according to Claim 6, in which one or more of the following parameters are processed as input variables of the system model: - a respective speed of the conveyor subsections (11-13, 21-23, 31-33); - a position of a respective piece good; - a size of a respective piece good; - a throughput of piece goods at the output (41) of the combining unit (40); - a collision in the combining unit (40).

8. Method according to Claim 7, in which the input variables are operating parameters of the system model.

9. Apparatus for computer-implemented configuration of a controlled drive application of a logistics system (1), wherein the logistics system (1) comprises a plurality of parallel-running conveyor sections (10, 20, 30) for piece goods that each lead to a combining unit (40) in the conveying direction (FR), each of the conveyor sections (10, 20, 30) consisting of a plurality of conveyor subsections (11-13, 21-23, 31-33) that are accelerated or decelerated by a respectively associated drive (11A-13A, 21A-23A, 31A-33A) under the control of a computing unit (60) of the apparatus in order to render the combining unit (40) able to combine the piece goods onto a single output conveyor section (50) at defined intervals, the computing unit (60) being designed to carry out the following steps: - determining a system model of the logistics system (1) on the basis of operating data of the logistics system (1), the operating data (BD) being available for a multiplicity of times in the operation of the logistics system and comprising, for each time, measured values from sensors (18S-28S) of the logistics system and manipulated variable changes; - determining a control function of the logistics system (1), which control function comprises at least configuration data (KD) for the drives (11A-13A, 21A-23A, 31A-33A), on the basis of the system model by specifying one or more performance features to be attained in order to perform at least one control operation in the system model, involving simulating the operating data (BD) for a multiplicity of time steps, wherein a reward quantity is ascertained for each time step, wherein the control operation is used as the control function, involving a predefined fitness function that aggregates the reward quantities for a multiplicity of time steps satisfying a predetermined criterion.

10. Apparatus according to Claim 9, said apparatus being designed to carry out the method according to one of Claims 2 to 8.

11. Logistics system (1) having a plurality of parallel-running conveyor sections (10, 20, 30) for piece goods that each lead to a combining unit (40) in the conveying direction (FR), each of the conveyor sections (10, 20, 30) consisting of a plurality of conveyor subsections (11-13, 21-23, 31-33) that are accelerated or decelerated by a respectively associated drive (11A-13A, 21A-23A, 31A-33A) under the control of a computing unit (60) in order to render the combining unit (40) able to combine the piece goods onto a single output conveyor section (50) at defined intervals, wherein the logistics system (1) comprises an apparatus according to Claim 9 or 10.

12. Computer program containing program code, which is stored on a nonvolatile, machine-readable carrier, for carrying out a method according to one of Claims 1 to 8 when the program code is executed on a computer.