Method for training multiple artificial neural networks to assign calls to elevator cars

Abstract

A method for training a neural network (3a, 3b) to assign a call (21) to a car (5a, 5b) of an elevator (1), the method comprising: in a series of simulation steps, simulating an environment (7) in which a first car (5a) and a second car (5b) move between floors (8) of a building (11) in response to a call indicating a desired floor, wherein each simulation step comprises: determining a current state (15) of the environment (7), the current state comprising a current position (17) of each car, a current call list (19) and a new call (21); inputting first input data (23a) encoding at least a portion of the current state into a first neural network (3a) configured to convert the first input data into a first output value (25a) indicative of a probability and / or a trend that the first car is assigned to a new call; inputting second input data (23b) encoding at least a portion of the current state into a second neural network (3b) configured to convert the second input data into a second output value indicative of a probability and / or a trend that a second car is assigned to a new call; determining the selected car (31) using the output value; assigning a new call to the selected car, wherein a reward value (35a, 35b) that quantifies the usefulness of the assignment (33) is determined; the neural network is trained using reward values from past simulation steps to increase the usefulness of assignments performed in future simulation steps.

Description

Technical Field

[0001] The invention relates to a method for training a plurality of artificial neural networks for assigning calls to elevator cars. The invention also relates to an elevator and a method for controlling an elevator, as well as to a data processing device, a computer program and a computer-readable medium for carrying out one or both of these methods. Background Art

[0002] An elevator may include different cars for transporting people and / or objects between different floors of a building along different shafts. Given a list of calls indicating the desired floors of a building, the controller of the elevator needs to be able to assign each call to one of the cars and determine the order of floors that the cars should stop at to complete the assigned calls in an efficient manner. This may also be called trip scheduling.

[0003] Especially in very high-rise buildings with many floors and multiple elevators operating simultaneously, solving the trip scheduling problem using traditional elevator control algorithms, which are usually based on a set of known rules, can become very complex. In such cases, machine learning models can be leveraged to make better control decisions. Such models are typically trained using reinforcement learning, where the actions of a simulated elevator (the "agent") are evaluated with rewards, and the algorithm controlling the agent (e.g., in the form of an artificial neural network) is optimized based on the rewards so that future control decisions are better than past control decisions.

[0004] CN 113479727 A discloses a method for training an artificial neural network to estimate control commands for controlling an elevator. Summary of the invention

[0005] It can be seen that the purpose of the present invention is to provide an improved machine learning model for controlling an elevator comprising two or more cars. Another purpose of the present invention may be to provide an improved method for controlling an elevator, a corresponding elevator, a corresponding data processing device, a corresponding computer program and a corresponding computer readable medium.

[0006] These objects are achieved by the subject-matter of the independent claims. Advantageous embodiments are defined in the dependent claims and in the following description and the drawings.

[0007] A first aspect of the present invention relates to a computer-implemented method for training multiple artificial neural networks to assign calls to elevator cars. The method comprises: (1) in a series of simulation steps, simulating an environment in which at least a first car and a second car of an elevator move along different vertical axes between different floors of a building in response to a call indicating a desired floor of the building, wherein each simulation step comprises: (1.1) determining a current state of the environment, the current state comprising a current position of each car relative to the floor, a current list of calls assigned to the car, and a new call assigned to one of the cars; (1.2) inputting first input data encoding at least a portion of the current state into a first artificial neural network, the first artificial neural network being configured to convert the first input data into a first output value, the first output value indicating a probability and / or tendency of the first car being assigned to the new call; (1.3) converting at least a portion of the current state into a first output value; A portion of the encoded second input data is input to a second artificial neural network, which is configured to convert the second input data into a second output value, the second output value indicating the probability and / or tendency of the second car being assigned to the new call; (1.4) using the first output value and the second output value, determining one of the cars as the selected car; (1.5) updating the environment by assigning the new call to the selected car, wherein a first reward value and a second reward value are determined, wherein each reward value quantifies the usefulness of the assignment; (2) training the first artificial neural network using training data including the first reward value from past simulation steps, and training the second artificial neural network using training data including the second reward value from past simulation steps to increase the usefulness of assignments performed in future simulation steps.

[0008] According to the invention, if the new call is assigned to the first car as the selected car, the first reward value is additionally increased or decreased (2.1). Additionally or alternatively, if the new call is assigned to the second car as the selected car, the second reward value is additionally increased or decreased (2.2). This has the effect of forcing the artificial neural networks to compete with each other, thereby generating a plurality of artificial neural networks that are trained to achieve efficient control of the elevator. The high efficiency of the elevator controller is particularly reflected in low waiting times for passengers and / or high transport capacity.

[0009] The environment can be viewed as a computer-computed space in which the (virtual) elevator takes actions. Each artificial neural network can be viewed as an agent (or a component of such an agent) in a multi-agent environment, i.e., a multi-agent reinforcement learning (MARL) framework, in which agents act independently of each other while collaborating towards the same goal. This is achieved by providing each agent with observations of at least part of the state of the environment and rewards associated with these observations.

[0010] For example, the simulation of the environment may include the following components: elevator simulation, traffic simulation, interaction between elevator simulation and traffic simulation. The elevator simulation may be configured with at least one of the following parameters: maximum speed, acceleration, door opening time, door closing time, minimum door opening time, car capacity. The traffic simulation may be performed based on a specific traffic pattern that determines the local and / or temporal distribution of calls. For example, each call may be characterized by a unique ID, a timestamp, a starting floor, and a destination floor. During the simulation, the traffic pattern may change. The simulation of the interaction between the elevator and the call (i.e., the passenger) may be configured with parameters such as waiting time (i.e., the time the passenger waits before entering the car), travel time (i.e., the time the passenger travels before leaving the car), or destination time (i.e., the total time required to complete the call).

[0011] In general, as described above, the agent can be configured to receive observations from the environment at each simulation step and perform actions based on the observations, such as assigning a new call to one of the cars and optionally controlling the car accordingly. In reaction, the environment can update its state and emit observations of its updated state along with a reward, which is then used to train the agent to improve the actions it performs in future simulation steps.

[0012] For example, since an elevator has cars that are arranged to move independently of each other, there may be many agents (ie, artificial neural networks).

[0013] The state of the environment can be considered as a combination of the elevator state and the traffic state. For example, the elevator state can be defined by at least one of the following parameters: the current floor of each car, the current direction of movement of each car, the current door state of each car (open, moving, closed), the current number of passengers inside each car, the next reachable floor of each car. For example, the traffic state can be defined by at least one of the following parameters: active calls from and / or to each floor, the current number of passengers inside each car, the current number of passengers waiting on each floor.

[0014] Each input data may encode at least a part of the current state in a format specifically suitable for artificial neural networks (e.g., in the form of a vector). The first input data and the second input data may encode the same part or different parts of the current state. In other words, the first input data and the second input data may be identical or different from each other. Encoding at least a part of the current state may also mean indicating a possible order of floors at which the corresponding car should stop in order to complete the corresponding current call and the new call. It is also possible that the two input data encode the complete current state.

[0015] The current position of each car may be the ID of the floor where the car is currently located or a nearby floor.

[0016] Each output value may be a value from a discrete and / or continuous range of values, such as a Q value or a competition value. However, each output value may also be a percentage between 0 and 1 or a Boolean value.

[0017] Each artificial neural network may be, for example, a multilayer perceptron (MLP), a convolutional neural network (CNN), a recurrent neural network (RNN), a long short-term memory (LSTM), a graph neural network (GNN), or a combination of at least two of these examples.

[0018] The agent implemented by each artificial neural network can be policy-based, value-based, or a combination of both (i.e., an actor-critic agent).

[0019] The selected car may be determined with a certain degree of randomness, for example by using an epsilon-greedy algorithm and / or by calculating a probability distribution based on the output values ​​and determining the selected car according to the probability distribution. The degree of randomness may vary (in particular decrease) as training proceeds, for example between consecutive training steps.

[0020] Each reward value can be positive, negative, or zero. For example, a positive reward value can represent a "more useful" allocation, while a negative reward value can represent a "less useful" allocation. Reward values ​​can be calculated at regular intervals, or only in certain circumstances, such as when the environment is observed to transition from a current state to a next state. The second reward value can be equal to or different from the first reward value.

[0021] The simulation and training steps can be performed in parallel and / or alternately.

[0022] The trained first artificial neural network and the trained second artificial neural network can be regarded as products directly obtained by the method.

[0023] A second aspect of the invention relates to a computer-implemented method for controlling an elevator, wherein the elevator comprises a first car and a second car, wherein the first car and the second car are arranged to be movable along different vertical axes between different floors of a building, and a sensor system, wherein the sensor system is adapted to provide sensor data indicating a current state of the elevator. The method comprises: receiving sensor data, wherein the sensor data comprises a current position of each car relative to a floor, a list of current calls assigned to the cars, and a new call assigned to one of the cars, wherein each call indicates a desired floor of a building; inputting first input data generated according to at least a portion of the sensor data into a first artificial neural network, the first artificial neural network being configured to convert the first input data into a first output value, the first output value indicating a probability and / or tendency of the first car being assigned to the new call, wherein the first artificial neural network has been trained using the above and below methods; inputting second input data generated according to at least a portion of the sensor data into a second artificial neural network, the second artificial neural network being configured to convert the second input data into a second output value, the second output value indicating a probability and / or tendency of the second car being assigned to the new call, wherein the second artificial neural network has been trained using the above and below methods; determining one of the cars as a selected car using the first output value and the second output value; and assigning the new call to the selected car. This enables high efficiency in the control of the elevator. The high efficiency of the elevator controller is particularly reflected in the low waiting time and / or high transportation capacity of passengers.

[0024] The first input data and the second input data may be generated from the same part or different parts of the sensor data. It is also possible that both input data are generated from the complete sensor data.

[0025] This means that first a plurality of artificial neural networks are trained by the method described above to assign calls to cars of an elevator, and then the trained networks are used to control an elevator having at least two cars. This results in a computer-implemented method for training a plurality of artificial neural networks to assign calls to cars of an elevator and for controlling an elevator; wherein the elevator comprises a first car and a second car and a sensor system, wherein the first car and the second car are arranged to be movable along different vertical axes between different floors of a building, and the sensor system is adapted to provide sensor data indicating the current state of the elevator. To train the plurality of artificial neural networks, the method comprises the following steps:

[0026] (1) In a series of simulation steps, an environment is simulated in which at least a first car and a second car of an elevator move along different vertical axes between different floors of a building in response to a call indicating a desired floor of the building, wherein each simulation step includes: (1.1) determining a current state of the environment, the current state including a current position of each car relative to the floor, a current list of calls assigned to the car, and a new call assigned to one of the cars; (1.2) inputting first input data encoding at least a portion of the current state into a first artificial neural network, the first artificial neural network being configured to convert the first input data into a first output value, the first output value indicating a probability and / or tendency of the first car being assigned to the new call; (1.3) inputting second input data encoding at least a portion of the current state into a second artificial neural network, the second artificial neural network being configured to convert the second input data into a second output value two output values, wherein the second output value indicates the probability and / or tendency of the second car being assigned to the new call; (1.4) using the first output value and the second output value, determining one of the cars as the selected car; (1.5) updating the environment by assigning the new call to the selected car, wherein a first reward value and a second reward value are determined, wherein each reward value quantifies the usefulness of the assignment; (2) training a first artificial neural network using training data including first reward values ​​from past simulation steps, and training a second artificial neural network using training data including second reward values ​​from past simulation steps to increase the usefulness of assignments performed in future simulation steps; (2.1) if the new call is assigned to the first car as the selected car, increase or decrease the first reward value (2.1); (2.2) additionally or alternatively, if the new call is assigned to the second car as the selected car, increase or decrease the second reward value (2.2).

[0027] To control the elevator, the method comprises the following steps:

[0028] (3) receiving sensor data, wherein the sensor data includes a current position of each car relative to a floor, a list of current calls assigned to the cars, and a new call assigned to one of the cars, wherein each call indicates a desired floor of a building; (4) inputting first input data generated based on at least a portion of the sensor data into a first artificial neural network, the first artificial neural network being configured to convert the first input data into a first output value, the first output value indicating a probability and / or tendency of the first car being assigned to the new call, wherein the first artificial neural network has been trained using the above-mentioned and below-mentioned methods; (4) inputting second input data generated based on at least a portion of the sensor data into a second artificial neural network, the second artificial neural network being configured to convert the second input data into a second output value, the second output value indicating a probability and / or tendency of the second car being assigned to the new call, wherein the second artificial neural network has been trained using the above-mentioned methods; (5) determining one of the cars as a selected car using the first output value and the second output value; and (6) assigning the new call to the selected car.

[0029] The above and below methods can be automatically executed by a processor, especially a processor of an elevator controller.

[0030] A third aspect of the invention relates to a data processing device comprising a processor configured to perform one or both of the above and following methods. The data processing device may comprise hardware and / or software modules. In addition to the processor, the data processing device may also comprise a memory and one or more data communication interfaces for data communication with peripheral devices. For example, the data processing device may be a (cloud) server, a PC, a laptop, an elevator group controller, an elevator controller, an elevator car controller or a combination of at least two of these examples.

[0031] A fourth aspect of the invention relates to an elevator. The elevator comprises: a first car and a second car, the first car and the second car being arranged to be movable along different vertical axes between different floors of a building; and a sensor system adapted to provide sensor data indicating the current state of the elevator, wherein the sensor data comprises the current position of each car relative to the floor, a list of current calls assigned to the cars and new calls assigned to one of the cars, wherein each call indicates a desired floor of the building; and data processing means as described above and below.

[0032] For example, the car can be arranged to be movable along different shafts of a building. In particular, each car can be arranged to move in its own shaft.

[0033] Other aspects of the present invention relate to a computer program comprising instructions which, when executed by a processor, cause the processor to perform one or both of the above and below methods, and to a computer-readable medium in which the computer program is stored.

[0034] The computer readable medium may be a volatile or non-volatile data storage device. For example, the computer readable medium may be a hard drive, a USB (Universal Serial Bus) storage device, a RAM (Random Access Memory), a ROM (Read Only Memory), an EPROM (Erasable Programmable Read Only Memory), an EEPROM (Electrically Erasable Programmable Read Only Memory), a flash memory, or a combination of at least two of these examples. The computer readable medium may also be a data communication network, such as the Internet or a cloud, for downloading program code.

[0035] It should be noted that features of the methods described above and below may also be features of the data processing apparatus, the computer program and the computer-readable medium (and vice versa).

[0036] Embodiments of the present invention may be considered to be based on the ideas and findings described below, without limiting the present invention.

[0037] According to an embodiment, the first reward value may be determined as the second reward value. In other words, the first reward value and the second reward value may be the same, which means that different artificial neural networks are trained using the same reward value. This has the effect of forcing the artificial neural networks to cooperate with each other to obtain better rewards in the future. Alternatively, the first reward value and the second reward value may be different from each other and / or may be determined by different reward functions.

[0038] According to an embodiment, the first reward value may be additionally increased or decreased by an amount proportional to the first output value. For example, the amount used to increase or decrease the first reward value may be a linear function of the first output value.

[0039] According to an embodiment, the second reward value may be additionally increased or decreased by an amount proportional to the second output value. For example, the amount used to increase or decrease the second reward value may be a linear function of the second output value.

[0040] According to an embodiment, each reward value may be a function of at least one of the following inputs determined during the updating of the environment: the energy consumption of the elevator, the average time required to complete each call. For example, reducing the energy consumption and / or reducing the average time may increase each reward value. Each reward value may be a function of different inputs. In this case, different inputs may be weighted with different weights. Thus, each artificial neural network may be trained to achieve the lowest possible energy consumption or the shortest possible average time, or a compromise between the two.

[0041] Additionally, each reward value may be a function of at least one of the following inputs: the number of completed calls, the number of passengers in each car. The number of completed calls may be the number of calls completed by the elevator since the end of the last simulation step and / or since the last state transition.

[0042] According to an embodiment, the training data of each artificial neural network may also include at least one of the following data from past simulation steps: at least a portion of the current state of the environment, at least a portion of the next state of the environment, an allocation, a first output value, a second output value. The next state may follow the current state. For example, such training data may be used to train each artificial neural network using a value-based learning algorithm (also known as Q-learning). The allocation resulting from the allocation of a new call to a selected car may be regarded as an action selected by the corresponding agent in a simulation step. Training may be performed in a series of training steps. In each training step, training data may be sampled from a certain number of records stored in a memory, each record including the above data from one of the past simulation steps.

[0043] According to an embodiment, the method for controlling an elevator may further include generating a control command to cause the selected car to complete a new call. The control command may be generated to control the movement of the selected car along one of the vertical axes, such as along the shaft of a building. The control command may be generated taking into account other calls assigned to the selected car. Examples of such control commands are "up", "down", "go to floor X" or "go to next floor". The control command may also cause the selected car to close and / or open doors and / or complete other calls assigned to it.

[0044] According to an embodiment, the first input data may encode at least the current position of the first car, the current call assigned to the first car, and the new call. Additionally, the first input data may encode the current position of the second car and / or the current call assigned to the second car.

[0045] According to an embodiment, the second input data may encode at least the current position of the second car, the current call assigned to the second car, and the new call. Additionally, the second input data may encode the current position of the first car and / or the current call assigned to the first car.

[0046] As described above, for example, the current state and / or sensor data may also include the current direction of movement (e.g., "up", "down", "parked") and / or the current occupancy (e.g., "full", "not full") of each car. In this case, the first input data may additionally encode the current direction of movement and / or the current occupancy of the first car (and optionally the second car), and / or the second input data may additionally encode the current direction of movement and / or the current occupancy of the second car (and optionally the first car).

[0047] In other words, each agent can see not only information related to its own state, but also the state of any other agent.

[0048] This can improve the accuracy of the method.

[0049] According to an embodiment, the selected car may be the car corresponding to the lowest or highest output value.

[0050] According to an embodiment, the processor of the data processing device can be configured to perform the method of the second aspect of the present invention according to an embodiment of generating control commands. In this case, the elevator can also include an actuator system, which is suitable for controlling the car according to the control commands generated by the data processing device. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Advantageous embodiments of the invention are described in more detail below with reference to the accompanying drawings. Neither the description nor the drawings should be interpreted as limiting the invention.

[0052] Figure 1 A block diagram showing a simulation of an elevator in a method according to an embodiment of the invention is shown.

[0053] Figure 2 A block diagram showing the training of an artificial neural network in a method according to an embodiment of the present invention is shown.

[0054] Figure 3 An elevator according to an embodiment of the invention is shown.

[0055] Figure 4 A data processing apparatus according to an embodiment of the present invention is shown.

[0056] The drawings are schematic diagrams only and are not drawn to scale. Identical reference numerals in the drawings denote identical features or features having the same effect. DETAILED DESCRIPTION

[0057] Figure 1 A simulation of an elevator 1 is shown as part of a training method for training a plurality of artificial neural networks 3a, 3b to assign calls to cars 5a, 5b of the elevator 1.

[0058] The elevator 1 is simulated in a virtual environment 7 in which a virtual first car 5a and a virtual second car 5b (or more than two virtual cars) of the elevator 1 move along different shafts 8 between different floors 9 of a virtual building 11 in response to calls from virtual passengers 13. Each call indicates a desired floor at which one of the cars 5a, 5b should stop.

[0059] The simulation is performed in a series of simulation steps. In each simulation step, the environment 7 determines its current state 15, including the current position 17 of each car 5a, 5b relative to the floor 9, the current call list 19 assigned to the cars 5a and 5b, and the new call 21 to be assigned to one of the cars 5a, 5b.

[0060] At least a portion of the current state 15 can be converted into first input data 23a and second input data 23b. For example, the first input data 23a can encode the current position 17 of the first car 5a, the current call assigned to the first car 5a, and the new call 21, while the second input data 23b can encode the current position 17 of the second car 5b, the current call assigned to the second car 5b, and the new call 21.

[0061] The current state 15 may include additional data, such as the current direction of movement or the current occupancy of each car 5a, 5b. In this case, the first input data 23a may also encode the current direction of movement and / or the current occupancy of the first car 5a, and the second input data 23b may also encode the current direction of movement and / or the current occupancy of the second car 5a.

[0062] Alternatively, the first input data 23a may encode the possible order of the desired floors that the first car 5a should stop at, and / or the second input data 23b may encode the possible order of the desired floors that the second car 5b should stop at. The possible order may be determined based on the current position 17, the current call, the new call 21, the current direction of movement of the respective cars 5a, 5b or the current occupancy, or a combination of at least two of these parameters.

[0063] In particular, each input data 23 a , 23 b may encode the complete current state 15 , ie, encode all the above-mentioned data included in the current state 15 .

[0064] The first input data 23a is then input into the first artificial neural network 3a, which is configured to convert the first input data 23a into a first output value 25a indicating the probability and / or tendency of the first car 5a being assigned to the new call 21.

[0065] In parallel, second input data 23b are input into the second artificial neural network 3b, which is configured to convert the second input data 23b into a second output value 25b indicating the probability and / or tendency of the second car 5b being assigned to the new call 21.

[0066] Each neural network 3a, 3b may include a plurality of hidden layers 27 having trainable parameters for converting respective input data 23a, 23b into respective output values ​​25a, 25b.

[0067] For example, the output values ​​25a, 25b may be Q values ​​and / or competition values.

[0068] Next, the output values ​​25a, 25b are input to the evaluation module 29, which analyzes the output values ​​25a and 25b to determine one of the cars 5a, 5b as the selected car 31. For example, the selected car 31 may be the car corresponding to the lowest output value. Here, the first output value 25a is the lowest output value. Therefore, the first car 5a is determined as the selected car 31. Alternatively, the car corresponding to the highest output value may be determined as the selected car 31.

[0069] During the training of the network 3a, 3b, the selected car 31 may be determined with a certain degree of randomness, for example using an epsilon-greedy algorithm.

[0070] The evaluation module 29 then assigns the new call 21 to the selected car 31. This assignment 33 enables the environment 7 to update its state, for example by moving the selected car 31 from its current position 17 according to the new call 21. By doing so, the environment 7 calculates a first reward value 35a and a second reward value 35b, for example for the average waiting and / or travel time of the passenger 13 and / or the energy consumption of the elevator 1, both of which quantify the usefulness of the assignment 33.

[0071] The two reward values ​​35a, 35b may be the same reward value. However, the reward values ​​35a, 35b may also be output by different reward functions. As described below, the reward values ​​35a, 35b are used to train the neural networks 3a, 3b.

[0072] The assignment 33 may also enable the environment 7 to determine a transition from the current state 15 to a next state 37, which may include the same type of data as the current state 15, such as the next position of each car 5a, 5b, an updated version of the current call list 31, and the next new call.

[0073] In this example, a record 39 of the data generated at each simulation step (eg, including the current state 15, the next state 37, the allocation 33, and the reward values ​​35a, 35b) may be stored in a replay buffer 41 (see Figure 2 ). The record 39 of past simulation steps can be used to generate training data 43 for training the neural network 3a, 3b.

[0074] Figure 2 The training part of the training method is shown. The training may include a series of training steps. In each training step, a batch of N recorded information 39 may be sampled and used as training data 43.

[0075] The sampling can be performed in such a way that the training data 43 used to train the first neural network 3a includes the first reward values ​​35a from past simulation steps and the training data 43 used to train the second neural network 3b includes the second reward values ​​35b from past simulation steps. The training data 43 of different neural networks 3a, 3b can be the same or different from each other.

[0076] The training data 43 may also include the following data from past simulation steps: at least a portion of the current state 15, at least a portion of the next state 37, the allocation 33 (as the selected action).

[0077] This training data can be used to train the neural networks 3a, 3b using a value-based learning algorithm ("Q-learning"). Alternatively, a policy-based learning algorithm or a combination of both can be used. Examples of suitable learning algorithms are "Deep Q Networks" or "Distributed Prioritized Experience Replay (ApeX)" implemented in RLlib.

[0078] In principle, training is performed in a series of training steps in which the optimizer 45 adjusts the weights of each neural network 3a, 3b according to the corresponding reward value 35a, 35b so that the allocation 33 performed in future simulation steps is more useful than the allocation performed in past simulation steps. For example, the optimization can be performed by back propagation using stochastic gradient descent. The training steps can be performed in parallel or alternately with the simulation steps until a global or local optimum is reached.

[0079] For example, as described above, the reward value 35a, 35b may be related to the average time required to complete each call and / or the energy consumption of the elevator 1. This forces the neural networks 3a, 3b to cooperate with each other.

[0080] Alternatively, if the new call 21 is assigned to the first car 5a, the first reward value 35a may be additionally reduced (or increased), and if the new call 21 is assigned to the second car 5b, the second reward value 35b may be additionally reduced or increased. The amount by which each reward value 35a, 35b is reduced (or increased) may be proportional to the corresponding output value 25a, 25b. This forces the neural networks 3a, 3b to compete with each other.

[0081] The trained neural networks 3a, 3b can be used in a real version of the elevator 1, such as Figure 3 shown.

[0082] Similar to the simulated version, the real version of the elevator 1 comprises a first car 5a and a second car 5b which are arranged to be movable along different shafts 8 between different floors 9 of a real building 11 to transport real passengers 13.

[0083] The elevator 1 further comprises a sensor system 47, an actuator system 49 (eg comprising an electric drive "M" for each car 5a, 5b) and a controller 51 as data processing means.

[0084] The sensor system 47 is adapted to generate sensor data 52 encoding the current state of the elevator 1. For example, the sensor data 53 may comprise the same type of data as the current state 15 of the environment 7.

[0085] The actuator system 49 is adapted to control the cars 5a, 5b according to control commands 55 generated by the controller 51, for example to accelerate and decelerate the cars 5a, 5b and to close and open the doors of the cars 5a, 5b.

[0086] The controller 51 comprises a memory 57 and a processor 59, wherein the processor 59 is configured to execute the following method for controlling the elevator 1 by executing a computer program stored in the memory 57, namely generating a control command 55 (see also Figure 4 ).

[0087] In a first step, sensor data 53 are received by the controller 51 .

[0088] In a second step, at least part of the sensor data 53 is converted into first input data 23a and second input data 23b. For example, the (real) input data 23a, 23b may comprise the same type of data as the (virtual) input data 23a, 23b generated by the environment 7.

[0089] In a third step, first input data 23a are input into the trained first neural network 3a. In parallel, second input data 23b are input into the trained second neural network 3b.

[0090] In a fourth step, the resulting output values ​​25a, 25b are input to an evaluation module 29 which analyses the resulting output values ​​to determine one of the cars 5a, 5b as the selected car 31. In this example, the selected car 31 is the car corresponding to the lowest output value, here the first car 5a.

[0091] In a fifth step, the evaluation module 29 assigns the new call 21 comprised in the sensor data 53 to the selected car 31 .

[0092] In a sixth step, the evaluation module 29 may generate a control command 55 to cause the selected car 31 to complete the new call 21 and, in addition, to complete any other calls currently assigned to the selected car 31 .

[0093] The processor 59 may additionally be configured to perform the above-mentioned training method.

[0094] The above modules may be software and / or hardware modules.

[0095] In summary, the bidding-based strategy implemented by the above-described method can enable elevators to transport passengers to their final destinations in the shortest possible time while consuming as little energy as possible.

[0096] Finally, it should be noted that terms such as "comprises", "comprising", "having" or "with" do not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. It should also be noted that features or steps described with reference to one of the above embodiments can also be used in combination with features and steps described with reference to any other embodiment described above. Reference signs in the claims should not be construed as limiting the scope of the claims.

Claims

1. A computer-implemented method for training a plurality of artificial neural networks (3a, 3b) to assign calls (21) to cars (5a, 5b) of an elevator (1), the method comprising: In a series of simulation steps, an environment (7) is simulated in which at least a first car (5a) and a second car (5b) of the elevator (1) move along different vertical axes between different floors (8) of the building (11) in response to a call indicating a desired floor of the building (11), wherein each simulation step comprises: determining a current state (15) of the environment (7), the current state (15) comprising a current position (17) of each car (5a, 5b) relative to the floor (8), a list (19) of current calls assigned to the cars (5a, 5b), and a new call (21) to be assigned to one of the cars (5a, 5b); inputting first input data (23a) encoding at least a portion of the current state (15) into a first artificial neural network (3a), the first artificial neural network (3a) being configured to convert the first input data (23a) into a first output value (25a), the first output value indicating a probability and / or tendency of the first car (5a) being assigned to the new call (21); inputting second input data (23b) encoding at least a portion of the current state (15) into a second artificial neural network (3b), the second artificial neural network being configured to convert the second input data (23b) into a second output value (25b), the second output value indicating a probability and / or tendency of the second car (5b) being assigned to the new call (21); Using the first output value (25a) and the second output value (25b), determining one of the cars (5a, 5b) as a selected car (31); updating the environment (7) by assigning the new call (21) to the selected car (31), wherein a first reward value (35a) and a second reward value (35b) are determined, wherein each reward value (35a, 35b) quantifies the usefulness of the assignment (33); training the first artificial neural network (3a) using training data (43) including first reward values ​​(35a) from past simulation steps, and training the second artificial neural network (3b) using training data (43) including second reward values ​​(35b) from past simulation steps to increase the usefulness of allocations (33) performed in future simulation steps; wherein if the new call (21) is assigned to the first car (5a) as the selected car (31), the first reward value (35a) is additionally increased or decreased, If the new call (21) is assigned to the second car (5b) which is the selected car (31), the second reward value (35b) is additionally increased or decreased.

2. The method according to claim 1, The first reward value (35a) is determined as the second reward value (35b).

3. The method according to claim 1 or 2, wherein the first reward value (35a) is additionally increased or decreased by an amount proportional to the first output value (25a); and / or The second reward value (35b) is additionally increased or decreased by an amount proportional to the second output value (25b).

4. The method according to one of the preceding claims, Each reward value (35a, 35b) is a function of at least one of the following inputs determined during updating of the environment (7): energy consumption of the elevator (1), average time required to complete each call.

5. The method according to one of the preceding claims, The training data (43) of each artificial neural network (3a, 3b) further comprises at least one of the following data from the past simulation step: at least a part of the current state (15) of the environment (7), at least a part of the next state (37) of the environment (7), the allocation (33), the first output value (25a), and the second output value (25b).

6. A computer-implemented method for controlling an elevator (1), wherein the elevator (1) comprises a first car (5a) and a second car (5b), the first car and the second car being arranged to be movable along different vertical axes between different floors (8) of a building (11), and a sensor system (47) adapted to provide sensor data (53) indicative of a current state (15) of the elevator (1), wherein the method comprises: receiving the sensor data (53), wherein the sensor data (53) includes a current position (17) of each car (5a, 5b) relative to the floor (8), a list (19) of current calls assigned to the cars (5a, 5b), and new calls (21) assigned to one of the cars (5a, 5b), wherein each call indicates a desired floor of the building (11); inputting first input data (23a) generated by at least a part of the sensor data (53) into a first artificial neural network (3a), the first artificial neural network (3a) being configured to convert the first input data (23a) into a first output value (25a), the first output value indicating a probability and / or a tendency that the first car (5a) is assigned to the new call (21), wherein the first artificial neural network (3a) has been trained using a method according to one of the preceding claims; inputting second input data (23b) generated by at least a part of the sensor data (53) into a second artificial neural network (3b), the second artificial neural network being configured to convert the second input data (23b) into a second output value (25b), the second output value indicating a probability and / or a tendency that the second car (5b) is assigned to the new call (21), wherein the second artificial neural network (3b) has been trained using a method according to one of the preceding claims; using the first output value (25a) and the second output value (25b) to determine one of the cars (5a, 5b) as a selected car (31); The new call (21) is assigned to the selected car (31).

7. The method according to claim 6, further comprising: A control command (55) is generated to cause the selected car (31) to complete the new call (21).

8. The method according to one of the preceding claims, wherein the first input data (23a) encode at least the current position (17) of the first car (5a), the current call assigned to the first car (5a) and the new call (21); and / or The second input data (23b) encode at least the current position (17) of the second car (5b), the current call assigned to the second car (5b), and the new call (21).

9. The method according to one of the preceding claims, The selected car (31) is the car (5a, 5b) corresponding to the lowest output value or the highest output value (25a, 25b).

10. A data processing device (51) comprising a processor (59) configured to perform at least one of the methods according to one of claims 1 to 9.

11. An elevator (1), comprising: a first car (5a) and a second car (5b), the first car and the second car being arranged to be movable along different vertical axes between different floors (8) of a building (11); a sensor system (47) adapted to provide sensor data (53) indicative of a current state (15) of the elevator (1), wherein the sensor data (53) comprises a current position (17) of each car (5a, 5b) relative to the floor (8), a list (19) of current calls assigned to the cars (5a, 5b), and new calls (21) assigned to one of the cars (5a, 5b), wherein each call indicates a desired floor of the building (11); The data processing device (51) according to claim 10.

12. Elevator (1) according to claim 11, wherein the processor (59) of the data processing device (51) is configured to execute the method according to claim 8; The elevator (1) further comprises an actuator system (49) adapted to control the cars (5a, 5b) according to control commands (55) generated by the data processing device (51).

13. A computer program comprising instructions which, when a processor (59) executes said program, cause said processor (59) to perform at least one of the methods according to one of claims 1 to 9.

14. A computer-readable medium comprising instructions which, when executed by a processor (59), cause the processor (59) to perform at least one of the methods according to one of claims 1 to 9.

Images