A method, device and storage medium for scheduling elevator groups

Abstract

This invention discloses an elevator group scheduling method, device, and storage medium. The method includes: identifying multiple elevators to be scheduled in a building; the building having multiple floors; upon receiving an elevator call signal, collecting various state information related to scheduling for each elevator in the current state to construct a state space in reinforcement learning; setting scheduling actions related to whether each elevator stops at a floor for each elevator to construct an action space in reinforcement learning; inputting the state space into a Q-learning model of reinforcement learning to learn the value of each scheduling action in the action space in the state space for reducing the waiting time of passengers on multiple floors, the travel time of passengers in multiple elevators, and the energy consumption of multiple elevators as a whole; executing the scheduling action with the highest value for each elevator to reduce the waiting time of passengers on multiple floors, the travel time of passengers in multiple elevators, and the energy consumption of multiple elevators as a whole. This improves the efficiency of elevator scheduling.

Description

Technical Field

[0001] The present invention relates to the field of elevator scheduling technology, and in particular to an elevator group scheduling method, equipment and storage medium. Background Technology

[0002] With the continuous advancement of urbanization, high-rise buildings are increasing. As a vertical transportation tool in modern buildings, the operating efficiency and scheduling strategies of elevators affect passenger waiting time and overall elevator operating energy efficiency.

[0003] Traditional elevator scheduling strategies typically employ simple, pre-defined rules, such as prioritizing the shortest waiting time or the shortest path. These methods have limited ability to adjust elevator scheduling strategies. In some cases, if elevators are scheduled according to the original strategy, it may result in long waiting times for passengers on floors, long waiting times within the elevator itself, and high elevator energy consumption, leading to low elevator operating efficiency and wasted resources. Summary of the Invention

[0004] This invention provides an elevator group scheduling method, device, and storage medium to improve the efficiency of elevator scheduling.

[0005] In a first aspect, embodiments of the present invention provide an elevator group scheduling method, comprising:

[0006] Identify multiple elevators in a building that need to be dispatched; the building has multiple floors.

[0007] Upon receiving an elevator call signal, various state information related to scheduling for each elevator in the current state is collected to construct the state space in reinforcement learning;

[0008] The elevators described herein are configured with scheduling actions related to whether or not they stop at the specified floors, in order to construct the action space in reinforcement learning;

[0009] The state space is input into a Q-learning model of reinforcement learning to learn the value of each scheduling action in the action space in terms of compressing the waiting time of passengers on multiple floors, the elevator travel time of passengers in multiple elevators, and the energy consumption of multiple elevators in the overall state space.

[0010] The highest-value scheduling action is executed for each of the elevators to reduce overall passenger waiting time on multiple floors, passenger travel time in multiple elevators, and energy consumption of multiple elevators.

[0011] Secondly, embodiments of the present invention also provide an elevator group scheduling device, comprising:

[0012] An elevator determination module is used to determine multiple elevators to be dispatched in a building; the building has multiple floors;

[0013] The state information acquisition module is used to collect various state information related to scheduling of each elevator in the current state when a call signal is received, so as to construct the state space in reinforcement learning.

[0014] The scheduling action setting module is used to set scheduling actions related to whether or not to stop at the floor for each of the elevators, so as to construct the action space in reinforcement learning;

[0015] The value learning module is used to input the state space into a Q-learning model of reinforcement learning to learn the value of each scheduling action in the action space in the state space for compressing the waiting time of passengers on multiple floors, the elevator travel time of passengers in multiple elevators, and the energy consumption of multiple elevators as a whole.

[0016] The scheduling action selection module is used to execute the highest-value scheduling action for each of the elevators, so as to reduce the waiting time of passengers on multiple floors, the riding time of passengers in multiple elevators, and the energy consumption of multiple elevators as a whole.

[0017] Thirdly, embodiments of the present invention also provide a computer device, the computer device comprising:

[0018] One or more processors;

[0019] Storage device for storing one or more programs;

[0020] When the one or more programs are executed by the one or more processors, the one or more processors implement the elevator group scheduling method provided in the first aspect of the present invention.

[0021] Fourthly, embodiments of the present invention also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the elevator group scheduling method as provided in the first aspect of the present invention.

[0022] Fifthly, embodiments of the present invention also provide a computer program product, the computer program product including a computer program, which, when executed by a processor, implements the elevator group scheduling method provided in the first aspect of the present invention.

[0023] In this embodiment of the invention, multiple elevators to be scheduled in a building are identified; the building has multiple floors; upon receiving an elevator call signal, various state information related to scheduling for each elevator in the current state is collected to construct a state space in reinforcement learning; scheduling actions related to whether or not the elevator stops at a floor are set for each elevator to construct an action space in reinforcement learning; the state space is input into the Q-learning model of reinforcement learning to learn the value of each scheduling action in the action space in terms of reducing the waiting time of passengers on multiple floors, the riding time of passengers in multiple elevators, and the energy consumption of multiple elevators as a whole; the scheduling action with the highest value is executed for each elevator to reduce the waiting time of passengers on multiple floors, the riding time of passengers in multiple elevators, and the energy consumption of multiple elevators as a whole. By collecting the elevator state information in real time, the specific operating status of each elevator at the current moment is obtained, providing accurate data support for subsequent scheduling decisions. Integrating various elevator state information into a state space provides a comprehensive and clear state description for the Q-learning model. Combining scheduling actions to form an action space clarifies various operational decisions related to elevator stops, ensuring the diversity and flexibility of scheduling strategies. By inputting the state space into the Q-learning model, the model can understand the effectiveness of the current scheduling strategy and select the optimal scheduling action, thereby improving elevator scheduling efficiency. By executing the most valuable scheduling action, the elevator scheduling system can select the optimal action plan at specific scheduling moments, thus reducing passenger waiting time, elevator travel time, and elevator system energy consumption globally. This achieves a globally optimal elevator scheduling effect, effectively improving the elevator scheduling system's operational efficiency, response speed, and energy management capabilities. Attached Figure Description

[0024] Figure 1 This is a flowchart of an elevator group scheduling method provided in Embodiment 1 of the present invention;

[0025] Figure 2 This is a schematic diagram of the state space structure provided in Embodiment 1 of the present invention;

[0026] Figure 3 This is a flowchart of an elevator group scheduling method provided in Embodiment 2 of the present invention;

[0027] Figure 4 This is a schematic diagram of the time division structure provided in Embodiment 2 of the present invention;

[0028] Figure 5 This is a structural block diagram of an elevator group scheduling device provided in Embodiment 3 of the present invention;

[0029] Figure 6 This is a schematic diagram of the structure of a computer device provided in Embodiment 4 of the present invention. Detailed Implementation

[0030] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0031] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be used interchangeably where appropriate so that the embodiments of the invention described herein can cover implementations in sequences other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0032] Example 1

[0033] See Figure 1 The diagram illustrates a flowchart of an elevator group scheduling method according to Embodiment 1 of the present invention. This embodiment is applicable to elevator group scheduling. The method can be executed by an elevator group scheduling device, which can be implemented in hardware and / or software and can be configured in a computer device. Figure 1 As shown, the method includes:

[0034] Step 101: Identify the multiple elevators in the building that need to be dispatched.

[0035] A building is a structure of a certain size, function, and structure, typically consisting of multiple floors, used to meet the needs of residential, office, commercial, or other activities. Building design not only focuses on the building's appearance and spatial layout but also needs to consider structural safety, comfort, convenience, and harmony with the surrounding environment. Depending on their purpose, buildings can be classified as residential buildings, office buildings, commercial buildings, and industrial buildings, among others.

[0036] An elevator is an electrically powered device used to vertically transport people or goods between floors in a building. Elevators typically consist of a car, control system, drive unit, and cables, enabling fast and smooth vertical movement. Depending on their purpose, elevators come in various types, including passenger elevators, freight elevators, and cargo elevators. Each type is designed and installed according to its load capacity, speed, and operating environment. In modern buildings, elevators are widely used in commercial buildings, residential buildings, hospitals, airports, and various other locations.

[0037] In this embodiment, the number of elevators in a building is usually limited. If it is uncertain which elevators need to be scheduled, the elevator scheduling system may waste energy, time, and space. By clearly identifying the elevators to be scheduled, the movement of each elevator can be rationally arranged, avoiding waste of resources.

[0038] Different elevators may be in different locations and have different operating states, such as stopped, idle, or running. Identifying the elevators to be dispatched helps optimize elevator dispatching decisions based on factors such as the elevator's current location, task completion status, and energy consumption.

[0039] Step 102: Upon receiving the elevator call signal, collect various state information of each elevator in the current state related to scheduling in order to construct the state space in reinforcement learning.

[0040] In this embodiment, the elevator scheduling problem is a dynamic, real-time decision-making problem, and the optimal strategy may differ under different time periods and conditions. Upon receiving an elevator call signal, various status information related to scheduling for each elevator in the current time state is collected in real time to understand the current status of each elevator, so as to provide accurate input for the reinforcement learning algorithm and thus effectively make elevator scheduling decisions.

[0041] The present tense refers to the current moment or real-time state, used to describe the state or behavior of things at a specific point in time. In this embodiment, the present tense emphasizes that when an elevator dispatching system receives a call signal, it needs to collect various current state information of the elevator. This state information is real-time and represents the elevator's operating status at the current moment. The purpose of using the present tense is to ensure that dispatching decisions reflect the passenger demand on the current floor and the elevator's operating status, thereby optimizing the elevator dispatching plan and improving overall efficiency.

[0042] Elevator call signals include external and internal call signals. When the elevator dispatching system receives a call signal, it enters the decision-making phase. This phase involves determining how the elevator dispatching system should select the appropriate action to perform to complete the task associated with the call signal. An internal call signal indicates that a passenger inside the elevator has pressed a floor button, requesting the elevator to go to their desired floor. An external call signal indicates that a passenger on a floor has pressed an elevator button, requesting the elevator to go up or down. Some external call signals only include the departure floor, while others include both the departure and destination floors; however, all internal call signals include the destination floor.

[0043] Reinforcement learning is a machine learning method that learns optimal decisions through the interaction between an agent and its environment. Its core lies in the agent's trial-and-error process, receiving feedback (rewards) from the results of each decision (action), and continuously adjusting its strategy to maximize cumulative rewards in the long run. In elevator scheduling, the agent selects actions based on the current elevator state, optimizing multiple objectives such as passenger waiting time and elevator energy consumption to ultimately achieve optimal overall scheduling.

[0044] An agent is an entity capable of making decisions and taking actions based on feedback from the environment. In this invention, the agent is the elevator scheduling system, responsible for selecting scheduling actions based on collected state information and ultimately executing the elevator scheduling task. The agent's goal is to optimize the overall performance of the elevator scheduling system, thereby reducing passenger waiting time, travel time, and elevator energy consumption.

[0045] The environment refers to the external system that the intelligent agent interacts with. The environment provides state information and rewards based on the agent's behavior (i.e., scheduling actions). In this invention, the environment consists of multiple floors and an elevator system within a building.

[0046] A state is a complete description of the environment by an agent at a given moment. The state space describes the current situation of the elevator group system, including the operation of the elevators and the needs of the passengers.

[0047] An action is a move taken by an agent in a given state. In this invention, the scheduling action refers to the behavior of the elevator based on the current state.

[0048] A reward is the feedback given by the environment after an agent performs a scheduling action. In this invention, the reward is a combination of multiple metrics to measure the performance of the elevator group scheduling system.

[0049] State space is a crucial concept in reinforcement learning, describing the environmental state that an agent perceives at each moment. In elevator scheduling, the state space contains all the information relevant to the scheduling decision, such as the location of each elevator, whether there is a call signal, and whether there are passengers. This information helps the reinforcement learning model evaluate each possible decision, thereby selecting the optimal action.

[0050] For example, constructing the state space in a reinforcement learning model, such as Figure 2 The diagram shown is a schematic representation of the state space structure. Figure 2 The state space is a matrix. Rows represent the first floor identifiers. Multiple first floor identifiers constitute the departure floors of the building. The departure floors in the building are floors 1, 2, ..., M, where M is a positive integer. The columns of the state space include the second floor identifiers, the up-call light time, the down-call light time, and the internal call identifiers, up-no-stop identifiers, down-no-stop identifiers, and stop identifiers for each elevator. Multiple second floor identifiers constitute the destination floors of the building. The state space contains information about N elevators, where N is a positive integer. Each group consists of 4 columns, and each group represents all the information of one elevator, arranged according to the elevator number. In each group, the first column represents the elevator's internal call identifier, the second column represents the elevator's up-no-stop identifier, the third column represents the elevator's down-no-stop identifier, and the fourth column represents the elevator's stop identifier.

[0051] The matrix composed of the departure layer and the destination layer is a binary matrix. The matrix composed of the uplighting time, the downlighting time, and the first floor identifier is a real number matrix. The information of N elevators and the first floor identifier form a binary matrix. The use of a binary matrix is ​​to speed up the computer's operation. The uplighting time and downlighting time are represented by a real number matrix because if the uplighting time or downlighting time is 400ms, representing it in binary would occupy many bits, which is not conducive to the computer's operation.

[0052] Initialize the state space, that is, assign the initial value of each element in the state space to 0.

[0053] Query the call signals received by each elevator in real time.

[0054] The floor number indicating the departure point of the elevator call signal is designated as the first floor identifier, and the floor number indicating the destination point is designated as the second floor identifier. The element at the intersection of the first and second floor identifiers is changed from 0 to 1. For example, if the element in the i-th row and j-th column is 1, it means that there is an elevator call signal with a departure floor of i and a destination floor of j. An element of 0 at the intersection represents that there is no passenger information for the elevator.

[0055] The duration of the external call signal in the elevator call signal statistics is used to record the time it takes from when a passenger requests to call the elevator until the elevator arrives at the passenger's location.

[0056] If the external call signal is an upward call for elevators, then the floor number from which the call signal originates is used as the first floor identifier, and the duration is the upward call light duration. The duration is assigned to the element that intersects the first floor identifier and the upward call light duration to facilitate the calculation of the time spent by the passenger calling the elevator.

[0057] If the external call signal is a downward call for the elevator, then the floor number from which the elevator call signal indicates departure is used as the first floor identifier, and the duration is the down-call lighting time. The duration is then assigned to the element at the intersection of the first floor identifier and the down-call lighting time.

[0058] For each elevator, when the call signal is an internal call signal, the floor number indicated by the internal call signal is used as the first floor identifier. The element of the intersection of the first floor identifier and the elevator's internal call identifier is changed from 0 to 1. This intersection element being 1 indicates that the passenger needs to go to the destination floor.

[0059] For each elevator, when the elevator is moving upwards, the floor number indicated by the call signal is designated as the first floor identifier, and the floor number where the elevator does not stop is designated as the upward non-stop identifier. The intersection of the first floor identifier and the upward non-stop identifier is changed from 0 to 1. When this intersection element is 1, it means that the elevator will not stop at the floor indicated by the call signal when it is moving upwards.

[0060] For each elevator, when the elevator is moving downwards, the floor number indicated by the call signal is designated as the first floor identifier, and the floor number where the elevator does not stop is designated as the non-stop-down identifier. The intersection of the first floor identifier and the non-stop-down identifier is changed from 0 to 1. When this intersection element is 1, it means that the elevator will not stop at the floor indicated by the call signal when moving downwards.

[0061] For each elevator, the floor number indicated by the call signal is used as the first floor identifier, and the floor number where the elevator stops is used as the stop identifier. The intersection of the first floor identifier and the elevator stop identifier is changed from 0 to 1. When this intersection element is 1, it means that the elevator will stop at the floor indicated by the call signal.

[0062] Step 103: Set up scheduling actions related to whether each elevator stops at a floor for each elevator in order to construct the action space in reinforcement learning.

[0063] In this embodiment, the goal of reinforcement learning is to maximize the efficiency of the elevator scheduling system through continuous training. In this process, the action space defines all possible scheduling actions that the elevator scheduling system can take. A scheduling action is the action that the elevator can perform when it receives a call signal, and the action space is the set of actions to choose from among these scheduling actions. By designing the action space to be related to whether the elevator stops at a specific floor, the reinforcement learning model can learn the optimal strategy based on the actual scheduling situation.

[0064] For example, under the constraints of the operating rules, the possible scheduling actions of each elevator are: stopping at the next floor with a call signal along the direction of movement, passing the next floor with a call signal along the direction of movement, and waiting on the current floor. The action space is then represented by a 3×N matrix, where an element with a value of 1 indicates that the elevator performs the corresponding scheduling action, and N represents the number of elevators.

[0065] In reinforcement learning, the action space refers to the set of all possible scheduling actions an agent can choose in each state. Reinforcement learning explores different scheduling actions and optimizes the decision-making process based on the reward of each action. In the context of elevator scheduling, the action space consists of different scheduling actions that determine the elevator's behavior when it receives a call signal.

[0066] Step 104: Input the state space into the Q-learning model of reinforcement learning to learn the value of each scheduling action in the action space in the state space for compressing the waiting time of passengers on multiple floors, the elevator travel time of passengers in multiple elevators, and the energy consumption of multiple elevators as a whole.

[0067] In this embodiment, the state space is input into a Q-learning model of reinforcement learning, allowing the elevator scheduling system to continuously optimize elevator scheduling decisions through self-learning, thereby improving the operational efficiency of the elevator group. The Q-learning model is a reinforcement learning method based on the state-action value function, which can evaluate and learn the optimal scheduling strategy across multiple possible states. By inputting the state space, the elevator scheduling system can capture the real-time state information of each elevator in the elevator group, forming a comprehensive basis for scheduling decisions. This enables the elevator scheduling system to dynamically adjust elevator scheduling actions to cope with real-time changes in building demand, maximizing the overall value of reducing passenger waiting time across multiple floors, passenger travel time in multiple elevators, and energy consumption across multiple elevators, rather than optimizing the performance of a single elevator.

[0068] Passenger waiting time is an important indicator for evaluating the service quality of an elevator system. Longer waiting times usually indicate that elevator scheduling is inefficient, resulting in a poor passenger experience.

[0069] The time spent riding an elevator reflects its operational efficiency and passenger experience. Excessive time indicates low elevator efficiency and may lead to empty runs or unreasonable stops.

[0070] Energy consumption is a component of elevator system operating costs. Excessive energy consumption not only increases operating costs but may also have adverse environmental impacts.

[0071] By comprehensively considering the waiting time of passengers on multiple floors, the riding time of passengers in multiple elevators, and the energy consumption of multiple elevators, elevator scheduling can achieve balanced optimization, meeting passenger needs while controlling energy consumption, thereby improving the overall operational efficiency of the elevator scheduling system.

[0072] Step 105: Perform the most valuable scheduling actions for each elevator to reduce the overall waiting time of passengers on multiple floors, the riding time of passengers in multiple elevators, and the energy consumption of multiple elevators.

[0073] In this embodiment, a reinforcement learning Q-learning model is used to comprehensively consider multiple factors, including: passenger waiting time on multiple floors, passenger travel time in multiple elevators, and energy consumption of multiple elevators. These three factors have a certain trade-off relationship. The goal of the reinforcement learning Q-learning model is to simultaneously reduce passenger waiting time and travel time while lowering energy consumption, thereby maximizing the overall efficiency of the elevator scheduling system. When executing specific scheduling actions, selecting the scheduling action with the highest execution value can most effectively shorten the overall passenger waiting time on multiple floors, passenger travel time in multiple elevators, and energy consumption of multiple elevators, achieving multi-dimensional comprehensive optimization.

[0074] The elevator scheduling system uses a Q-learning model based on reinforcement learning to learn and evaluate the effects of different scheduling actions. This reduces passenger waiting time, elevator travel time, and elevator energy consumption. Executing the optimal scheduling action improves the overall efficiency of the elevator group, ensuring each elevator arrives at the appropriate floor at the right time and minimizing resource waste.

[0075] In this embodiment of the invention, multiple elevators to be scheduled in a building are identified; the building has multiple floors; upon receiving an elevator call signal, various state information related to scheduling for each elevator in the current state is collected to construct a state space in reinforcement learning; scheduling actions related to whether or not the elevator stops at a floor are set for each elevator to construct an action space in reinforcement learning; the state space is input into the Q-learning model of reinforcement learning to learn the value of each scheduling action in the action space in terms of reducing the waiting time of passengers on multiple floors, the riding time of passengers in multiple elevators, and the energy consumption of multiple elevators as a whole; the scheduling action with the highest value is executed for each elevator to reduce the waiting time of passengers on multiple floors, the riding time of passengers in multiple elevators, and the energy consumption of multiple elevators as a whole. By collecting the elevator state information in real time, the specific operating status of each elevator at the current moment is obtained, providing accurate data support for subsequent scheduling decisions. Integrating various elevator state information into a state space provides a comprehensive and clear state description for the Q-learning model. Combining scheduling actions to form an action space clarifies various operational decisions related to elevator stops, ensuring the diversity and flexibility of scheduling strategies. By inputting the state space into the Q-learning model, the model can understand the effectiveness of the current scheduling strategy and select the optimal scheduling action, thereby improving elevator scheduling efficiency. By executing the most valuable scheduling action, the elevator scheduling system can select the optimal action plan at specific scheduling moments, thus reducing passenger waiting time, elevator travel time, and elevator system energy consumption globally. This achieves a globally optimal elevator scheduling effect, effectively improving the elevator scheduling system's operational efficiency, response speed, and energy management capabilities.

[0076] Example 2

[0077] Figure 3 This is a flowchart of an elevator group scheduling method provided in Embodiment 2 of the present invention. Based on Embodiment 1, this embodiment details the offline update process of the Q-learning model of reinforcement learning, such as... Figure 3 As shown, the method includes:

[0078] Step 301: Identify the multiple elevators in the building that need to be dispatched.

[0079] Step 302: Upon receiving the elevator call signal, collect various state information of each elevator in the current state related to scheduling in order to construct the state space in reinforcement learning.

[0080] Step 303: Set scheduling actions related to whether each elevator stops at a floor for each elevator in order to construct the action space in reinforcement learning.

[0081] Step 304: Input the state space into the Q-learning model of reinforcement learning to learn the value of each scheduling action in the action space in the state space for compressing the waiting time of passengers on multiple floors, the elevator travel time of passengers in multiple elevators, and the energy consumption of multiple elevators as a whole.

[0082] Step 305: Perform the most valuable scheduling actions for each elevator to reduce the overall waiting time of passengers on multiple floors, the riding time of passengers in multiple elevators, and the energy consumption of multiple elevators.

[0083] Step 306: Within the time period between the previous scheduling action and the current scheduling action, divide the time interval into multiple time intervals based on events that occur in multiple elevators and are unrelated to the elevator call signal.

[0084] In this embodiment, multiple time intervals are established by dividing the events that occur in multiple elevators during the time period between the previous scheduling action and the current scheduling action, which are unrelated to the elevator call signal. This effectively subdivides the dynamic changes that occur during elevator scheduling, understands and monitors the performance of each elevator in different time periods, identifies potential peak periods or inefficient situations, and adjusts scheduling actions based on this information, thereby optimizing the scheduling strategy and improving overall efficiency.

[0085] For example, such as Figure 4 The diagram illustrates the time-division structure. In the diagram, t and t' are both decision events. t represents the time of the previous scheduling action, and t' represents the time of the current scheduling action. t1 and t2 are non-decision events unrelated to the elevator call signal. Non-decision events typically refer to events that occur during elevator operation that are not directly related to a passenger pressing the call button, such as a passenger arriving at their destination floor and leaving the elevator, or an elevator malfunction. Based on the number of non-decision events, the time period between the previous and current scheduling actions is divided into multiple time intervals: PR1, PR2, and PR3. Considering non-decision events ensures that each passenger's time is included in the calculation. When passengers are boarding or alighting, their rewards must be included; otherwise, passenger data will be lost.

[0086] The elevator group control problem can be viewed as a sequential decision-making process. The state points of the sequence are mainly composed of discrete event points and decision event points. Discrete event points are passengers going up or down the elevator, while decision event points vary depending on the action space settings. However, the time intervals between decision events are not fixed, meaning it is a decision problem occurring in continuous time. Therefore, a continuous-time extension of MDP—the Semi-Markov Decision Process (SMDP)—needs to be introduced.

[0087] Semi-Markov Decision Processes (SMDPs) are an extended Markov Decision Process (MDP) designed to handle decision problems with non-fixed durations. In SMDPs, state transitions depend not only on the current state and actions but also on a time factor; that is, the duration of each state is stochastic. Unlike traditional MDPs where each state transition is instantaneous, SMDPs allow a state to remain in a particular state for a certain period until a transition is triggered. This allows SMDPs to better model complex decision processes that require consideration of time delays and uncertainties, such as elevator scheduling problems.

[0088] Step 307: Calculate the first index value representing the waiting time of passengers in the building in each time interval.

[0089] In this embodiment, calculating the first metric representing passenger waiting time on each floor is to more accurately measure passenger waiting experience during elevator scheduling. This process calculates passenger waiting time within each time interval to assess how the elevator scheduling system affects passenger waiting time in actual operation, thereby providing data support for scheduling optimization. This first metric provides feedback to the Q-learning model of reinforcement learning, helping it learn how to schedule elevators more effectively to reduce passenger waiting time, thereby improving the overall efficiency of the system and passenger experience.

[0090] Specifically, within each time interval, the time it takes for passengers to arrive at the elevator lobby on each floor is subtracted from the end time of that interval to obtain the first interval time for passengers waiting for the elevator. The first interval time is then used as the base and a preset adjustment coefficient as the exponent to perform a power operation to obtain the second interval time. The second interval times for each passenger are then summed to obtain the first index value representing the waiting time of passengers in the building. Here, the adjustment coefficient is a natural number.

[0091] Step 308: Calculate the second index value representing the elevator travel time of passengers in multiple elevators in each time interval.

[0092] In this embodiment, the overall efficiency of the elevator scheduling system is optimized by calculating a second index value representing the travel time of passengers in multiple elevators. Travel time refers to the time from when a passenger enters the elevator to when the elevator reaches its destination floor, and it is generally related to the passenger's travel experience and the elevator's operating efficiency. By analyzing and calculating the travel time within each time interval, we can gain a deeper understanding of the efficiency performance under different elevator operating conditions, thereby finding the optimal strategy for optimizing elevator scheduling. This second index value helps the Q-learning model of reinforcement learning accurately assess the impact of different scheduling actions on the travel time of passengers in the elevator.

[0093] Specifically, within each time interval, the difference between the end time of the time interval and the time it takes for passengers to enter the elevator car is used as the third interval time. The third interval time is then used as the base and a preset adjustment coefficient as the exponent to perform a power operation to obtain the fourth interval time. The fourth interval times for each passenger are then summed to obtain a second index value representing the travel time of passengers in multiple elevators. The adjustment coefficient is a natural number.

[0094] Step 309: Calculate the third index value representing the energy consumption of multiple elevators in each time interval.

[0095] In this embodiment, calculating the third index value representing the energy consumption of multiple elevators aims to optimize resource utilization and reduce energy consumption during elevator group scheduling, thereby improving the overall system efficiency. Elevators frequently stop at various floors during service, consuming electricity each time. Statistical analysis of these elevator energy consumptions helps the Q-learning model of reinforcement learning find more efficient and energy-saving scheduling strategies over a long period, thus improving the overall effectiveness of the elevator group scheduling system.

[0096] Specifically, in each time interval, the number of times the multiple elevators stop at each floor is added together to obtain the total number of stops. The preset energy consumption coefficient is then multiplied by the total number of stops to obtain the third index value representing the energy consumption of the multiple elevators.

[0097] Step 310: Merge the first indicator value, the second indicator value, and the third indicator value into the target indicator value.

[0098] In this embodiment, the purpose of merging the first indicator value (passenger waiting time), the second indicator value (elevation time), and the third indicator value (energy consumption) into a target indicator value in elevator group scheduling is to optimize the overall efficiency of elevator scheduling and balance the contradictions between different factors. The target indicator value is negatively correlated with the first, second, and third indicator values. This negative correlation means that the higher the first, second, and third indicator values ​​are, the lower the target indicator value will be. To maximize the target indicator value, the elevator scheduling system needs to balance passenger waiting time, elevator time, and energy consumption.

[0099] Specifically, the product of the first indicator value and the preset first weight, the product of the second indicator value and the preset second weight, and the product of the third indicator value and the preset third weight are added together to obtain the candidate indicator value. The negative number of the candidate indicator value is then taken to obtain the target indicator value.

[0100] Step 311: Integrate the product of the preset discount factor and the target index value over time within the time period to obtain the incentive value.

[0101] In this embodiment, the long-term value of each scheduling action is evaluated by weighing past decisions and future expected effects over time. A discount factor is used to control the impact of time delay on decision-making, ensuring that the reinforcement learning model not only focuses on current rewards but also considers potential future returns. Through integration, the elevator scheduling system can dynamically calculate the comprehensive impact of scheduling actions on target indicators (such as passenger waiting time, elevator travel time, and energy consumption) over the entire time period, enabling the Q-learning model of reinforcement learning to learn the optimal scheduling strategy.

[0102] For example, the incentive value is represented as:

[0103]

[0104] In the formula, R(s,a) is the excitation value, and t last t is the time of the last scheduled action. current Let (t, t1), (t1, t2), ..., (t...) be the times when the scheduling action is currently being executed. n, t′) are all time intervals, e -β(τ-t) The discount factor is preset, unlike the fixed discount factor in traditional reinforcement learning. It depends on the time interval; the longer the time interval, the greater the discount. τ For the target indicator value, Let τ be the time it takes for passengers to arrive at the elevator lobby on the floor, τ be the end time of the time interval, and HP be the total number of passengers in the waiting lobby. Let n be the time when passengers enter the elevator car, CP be the total number of passengers inside the car, and n be the total number of passengers inside the car. e Let w1 represent the number of stops of elevator e, where E is the set of elevators, w2 is the first weight, w3 is the second weight, and w3 is the third weight. For the first interval time, For the second time interval, The first indicator value, For the third time interval, This is the fourth time interval. The second indicator value is C, which is the preset energy consumption coefficient. ∑ e∈E n e Let C∑ be the total number of stops.e∈E n e The third indicator value is i, which is the adjustment coefficient. i = 0, 1, or 2. When i = 0, the time for passengers to arrive at the waiting hall on the floor and the time for passengers to enter the elevator car are both calculated as -1. When i = 1, the target indicator value is negatively correlated with the first and second indicator values. When i = 2, the target indicator value is negatively correlated with the squares of the first and second indicator values. When i = 2, it is to strengthen the influence of passenger waiting time and passenger elevator travel time in the building.

[0105] In one embodiment of the present invention, passenger distribution information in multiple floors and multiple elevators is collected; based on the distribution information, a proportional-integral-derivative (PID) control algorithm is used to dynamically adjust the first weight, the second weight, and the third weight; or, in a deep deterministic policy gradient framework (DDPG), the first weight, the second weight, and the third weight are updated with the optimization objective of reducing the overall waiting time of passengers in multiple floors, the elevator travel time of passengers in multiple elevators, and the energy consumption of multiple elevators. Through the above methods, the weight parameters can be dynamically adjusted based on the distribution information of each building, thereby better scheduling and allocating elevators.

[0106] The Proportional-Integral-Derivative (PID) control algorithm dynamically adjusts the system response by calculating the error in real time and dividing it into proportional, integral, and derivative terms. In elevator group scheduling, the PID algorithm dynamically adjusts the first, second, and third weights using real-time data on floor and passenger distribution within the elevators, employing the proportional term (reflecting the current error), the integral term (reflecting the accumulated error), and the derivative term (reflecting the trend of error change). Specifically, the error can include the following:

[0107] 1. Position Error: The difference between the elevator's current floor and the target floor. For example, if the elevator is on the 3rd floor and the target floor is the 5th floor, the error is 2 floors.

[0108] 2. Time Error: In elevator dispatching systems, error may represent the difference between the passenger's waiting time and the expected target time. For example, if a passenger waits for 5 minutes in front of the elevator, while the expected waiting time is 3 minutes, then the error is 2 minutes.

[0109] 3. Energy Consumption Error: The difference between the elevator's actual energy consumption and the expected energy consumption. For example, the system expects to consume a certain amount of electricity each time it is scheduled; if the consumption exceeds the expectation, this error will also be taken into account.

[0110] The Deep Deterministic Policy Gradient Framework (DDPG) is a reinforcement learning algorithm for optimization problems in a continuous action space. In elevator group scheduling, DDPG aims to reduce passenger waiting time, travel time, and elevator energy consumption across multiple floors. Through exploration and utilization, it continuously updates the values ​​of the first, second, and third weights. DDPG trains a policy network and a value network to dynamically adjust these first, second, and third weights to achieve optimal scheduling results. DDPG can handle complex environments and a continuous adjustment space, ensuring that the first, second, and third weights are dynamically adjusted appropriately under constantly changing scheduling requirements.

[0111] Step 312: Update the Q-learning model of reinforcement learning based on the incentive value.

[0112] In this embodiment, the Q-learning model of reinforcement learning optimizes the overall scheduling effect by learning the impact of different scheduling actions on the state of the elevator scheduling system. To achieve this goal, the core of reinforcement learning lies in adjusting decision-making strategies through feedback learning, thereby gradually improving the overall performance of the elevator scheduling system (such as passenger waiting time, elevator travel time, and elevator energy consumption). After each scheduling action is executed, the scheduling effect is evaluated based on the incentive value (i.e., the target indicator value obtained by fusing multiple indicators) in the current state. The incentive value reflects the contribution of the current scheduling action to the overall optimization goal; a higher incentive value indicates better elevator scheduling system performance. The Q-learning model of reinforcement learning is updated based on the incentive value to make future decisions more effective.

[0113] For example, based on the current incentive value, the Q-value (i.e., the effect of the scheduling action) is adjusted using the Q-learning update rule. In Q-learning, the Q-value represents the value of a specific state-action pair, indicating the expected reward obtainable by performing a scheduling action in the current state. The specific formula for the Q-learning update rule is as follows:

[0114]

[0115] Where Q(s) t a t ) is based on the previous state space s t Execute scheduling action a t The Q-value is given by α, the learning rate is given by R(s,a) and the discount factor is given by γ. Let s be the current state space t+1 The maximum Q value.

[0116] Updating the Q-learning model based on the incentive values ​​essentially utilizes the update rule formula of Q-learning in reinforcement learning algorithms to help the elevator scheduling system learn the optimal scheduling strategy. In this way, the reinforcement learning Q-learning model can continuously adjust its decision rules, thereby optimizing the overall performance of the elevator scheduling system. After each scheduling action, the reinforcement learning Q-learning model evaluates the effectiveness of the current strategy, using incentive value feedback to guide the learning process, enabling future scheduling decisions to better reduce passenger waiting time, improve elevator efficiency, and lower energy consumption.

[0117] In this embodiment of the invention, multiple elevators to be scheduled in a building are identified; the building has multiple floors; upon receiving an elevator call signal, various state information related to scheduling for each elevator in the current state is collected to construct a state space in reinforcement learning; scheduling actions related to whether or not the elevator stops at a floor are set for each elevator to construct an action space in reinforcement learning; the state space is input into the Q-learning model of reinforcement learning to learn the value of each scheduling action in the action space in the state space for reducing the waiting time of passengers on multiple floors, the riding time of passengers in multiple elevators, and the energy consumption of multiple elevators as a whole; the scheduling action with the highest value is executed for each elevator to reduce the waiting time of passengers on multiple floors, the riding time of passengers in multiple elevators, and the energy consumption of multiple elevators as a whole. Within the time period between the previous scheduling action and the current scheduling action, multiple time intervals are divided based on events occurring in multiple elevators that are unrelated to the call signal. In each time interval, a first indicator value representing the waiting time of passengers on each floor is calculated. In each time interval, a second indicator value representing the elevator travel time of passengers in multiple elevators is calculated. In each time interval, a third indicator value representing the energy consumption of multiple elevators is calculated. The first, second, and third indicator values ​​are then merged into a target indicator value. The target indicator value is negatively correlated with all three indicators. Within the time period, the product of a preset discount factor and the target indicator value is integrated over time to obtain an incentive value. The Q-learning model of reinforcement learning is updated based on this incentive value. By dividing the time period, the impact of various events triggered by non-call signals on the scheduling strategy during elevator scheduling is captured. Multi-dimensional indicators such as passenger waiting time, elevator travel time, and elevator energy consumption are dynamically calculated and integrated to form a comprehensive and optimized scheduling strategy, achieving a balanced scheduling strategy and ensuring coordination between different objectives. By calculating incentive values, the impact of each scheduling decision on future elevator scheduling efficiency can be more accurately measured, further guiding the system's long-term scheduling decisions. The Q-learning model of reinforcement learning is updated based on these incentive values, enabling continuous optimization of the elevator scheduling strategy over the long term, improving overall efficiency, while also considering energy consumption and passenger experience, ultimately achieving more intelligent and efficient elevator scheduling.

[0118] Example 3

[0119] Figure 5 This is a schematic diagram of the structure of an elevator group dispatching device provided in Embodiment 3 of the present invention, as shown below. Figure 5 As shown, the device includes:

[0120] Elevator determination module 501 is used to determine multiple elevators to be dispatched in a building; the building has multiple floors;

[0121] The state information acquisition module 502 is used to acquire various state information related to scheduling of each elevator in the current state when a call signal is received, so as to construct the state space in reinforcement learning.

[0122] The scheduling action setting module 503 is used to set scheduling actions related to whether or not the elevator stops at the floor for each of the elevators, so as to construct the action space in reinforcement learning.

[0123] The value learning module 504 is used to input the state space into a Q-learning model of reinforcement learning to learn the value of each scheduling action in the action space in the state space for compressing the waiting time of passengers on multiple floors, the elevator travel time of passengers in multiple elevators, and the energy consumption of multiple elevators as a whole.

[0124] The scheduling action selection module 505 is used to execute the scheduling action with the highest value for each of the elevators, so as to reduce the waiting time of passengers on multiple floors, the riding time of passengers in multiple elevators, and the energy consumption of multiple elevators as a whole.

[0125] In one embodiment of the present invention, the status information acquisition module 502 includes:

[0126] The state space construction module is used to construct the state space in the reinforcement learning model; the state space is a matrix, with rows being the first floor identifier, and columns including the second floor identifier, the up call light-on time, the down call light-on time, and the internal call identifier, the up-going non-stop identifier, the down-going non-stop identifier, and the stop identifier for each of the elevators.

[0127] The elevator call signal query module is used to query the elevator call signals received by each of the elevators in real time.

[0128] The floor information calibration module is used to take the floor number from which the elevator call signal indicates departure as the first floor identifier and the floor number to which the elevator call signal indicates arrival as the second floor identifier, and modify the element of the intersection of the first floor identifier and the second floor identifier from 0 to 1.

[0129] The external call duration statistics module is used to count the duration for which the external call signal is active in the elevator call signals;

[0130] The call duration calibration module is used to assign the duration to the element at the intersection of the first floor identifier and the call duration if the external call signal is an upward call for elevators.

[0131] The call duration calibration module is used to assign the duration to the element at the intersection of the first floor identifier and the call duration if the external call signal is a call to the elevator downwards.

[0132] The internal call signal calibration module is used to, for each of the elevators, when the elevator call signal is an internal call signal, take the floor number indicated by the internal call signal as the first floor identifier, and modify the element of the intersection of the first floor identifier and the internal call identifier of the elevator from 0 to 1.

[0133] The elevator up-going non-stop module is used to, for each of the elevators, when the elevator is moving upward, use the floor number indicated by the elevator call signal as the first floor identifier and the floor number where the elevator does not stop as the up-going non-stop identifier, and modify the element of the intersection of the first floor identifier and the up-going non-stop identifier of the elevator from 0 to 1.

[0134] The elevator downhill non-stop module is used to, for each of the elevators, when the elevator is moving downhill, use the floor number indicated by the elevator call signal as the first floor identifier and the floor number where the elevator does not stop as the downhill non-stop identifier, and modify the element of the intersection of the first floor identifier and the downhill non-stop identifier of the elevator from 0 to 1.

[0135] The elevator stopping module is used to modify the element of the intersection of the first floor identifier and the elevator's stopping identifier from 0 to 1 for each of the elevators, using the floor number indicated by the elevator call signal as the first floor identifier and the floor number where the elevator stops as the stopping identifier.

[0136] In one embodiment of the present invention, the scheduling action includes stopping at the next floor with an elevator call signal along the direction of movement, passing the next floor with an elevator call signal along the direction of movement, and waiting on the current floor.

[0137] In one embodiment of the present invention, the device further includes:

[0138] The time interval division module is used to divide the time period between the previous execution of the scheduling action and the current execution of the scheduling action into multiple time intervals based on events that occur in multiple elevators and are unrelated to the elevator call signal.

[0139] The first indicator value calculation module is used to calculate a first indicator value representing the waiting time of passengers in the building in each of the time intervals.

[0140] The second index value calculation module is used to calculate a second index value representing the elevator travel time of passengers in the multiple elevators in each of the time intervals.

[0141] The third indicator value calculation module is used to calculate a third indicator value representing the energy consumption of the multiple elevators in each of the time intervals.

[0142] The target indicator value acquisition module is used to merge the first indicator value, the second indicator value, and the third indicator value into a target indicator value; the target indicator value is negatively correlated with the first indicator value, the second indicator value, and the third indicator value.

[0143] The incentive value calculation module is used to integrate the product between the preset discount factor and the target index value over time within the time period to obtain the incentive value.

[0144] The model update module is used to update the Q-learning model of the reinforcement learning based on the incentive value.

[0145] In one embodiment of the present invention, the first index value calculation module includes:

[0146] The first interval calculation module is used to subtract the time it takes for the passenger to arrive at the elevator lobby on the floor from the end time of each time interval to obtain the first interval time for the passenger to wait for the elevator.

[0147] The second interval calculation module is used to perform a power operation with the first interval time as the base and a preset adjustment coefficient as the exponent to obtain the second interval time.

[0148] The first index value summation module is used to sum the second interval time corresponding to each passenger to obtain a first index value that represents the waiting time of passengers in the building.

[0149] The second indicator value calculation module includes:

[0150] The third interval calculation module is used to calculate the difference between the end time of the time interval and the time when the passenger enters the elevator car, and use the difference as the third interval time.

[0151] The fourth interval calculation module is used to perform a power operation with the third interval time as the base and a preset adjustment coefficient as the exponent to obtain the fourth interval time;

[0152] The second index value summing module is used to sum the fourth interval time corresponding to each passenger to obtain a second index value representing the elevator travel time of passengers in multiple elevators.

[0153] The adjustment coefficient is a natural number.

[0154] In one embodiment of the present invention, the third index value calculation module includes:

[0155] The total number of stops calculation module is used to add up the number of times the multiple elevators stop at the floors in each of the time intervals to obtain the total number of stops;

[0156] The third indicator value acquisition module is used to multiply the preset energy consumption coefficient by the total number of stops to obtain a third indicator value that characterizes the energy consumption of the multiple elevators.

[0157] In one embodiment of the present invention, the target indicator value acquisition module includes:

[0158] The candidate indicator value acquisition module is used to add the product between the first indicator value and the preset first weight, the product between the second indicator value and the preset second weight, and the product between the third indicator value and the preset third weight to obtain the candidate indicator value.

[0159] The target indicator value calculation module is used to take the opposite of the candidate indicator value to obtain the target indicator value.

[0160] In one embodiment of the present invention, it further includes:

[0161] The distribution information acquisition module is used to collect the distribution information of passengers in the floors and multiple elevators.

[0162] The weight adjustment module is used to dynamically adjust the first weight, the second weight, and the third weight based on the distribution information using a proportional-integral-derivative control algorithm.

[0163] or,

[0164] The weight update module is used to update the first weight, the second weight, and the third weight in a deep deterministic policy gradient framework with the optimization objective of compressing the waiting time of passengers on multiple floors, the elevator travel time of passengers in multiple elevators, and the energy consumption of multiple elevators as a whole.

[0165] The elevator group scheduling device provided in the embodiments of the present invention can execute the elevator group scheduling method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of executing the elevator group scheduling method.

[0166] Example 4

[0167] See Figure 6 This diagram illustrates a structural schematic of a computer device according to an embodiment of the present invention. The term "computer device" is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, blade servers, mainframe computers, and other suitable computers. The computer device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0168] like Figure 6 As shown, the computer device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer programs stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the computer device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0169] Multiple components in computer device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of monitors, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows computer device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0170] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as elevator group scheduling methods.

[0171] In some embodiments, the elevator group scheduling method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on computer device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the elevator group scheduling method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the elevator group scheduling method by any other suitable means (e.g., by means of firmware).

[0172] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0173] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0174] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0175] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer device having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0176] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0177] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0178] Example 5

[0179] This invention also provides a computer program product, which includes a computer program that, when executed by a processor, implements the elevator group scheduling method provided in any embodiment of this invention.

[0180] In implementing the computer program product, computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0181] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0182] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method of elevator group dispatching, characterized in that, The method comprises: determining a plurality of elevators to be dispatched in a building; the building has a plurality of floors; when receiving a call signal, collecting various state information of each of the elevators in the present tense related to dispatching to construct a state space in reinforcement learning; setting a dispatch action related to whether to stop at the floor for each of the elevators to construct an action space in reinforcement learning; inputting the state space into a Q-learning model of reinforcement learning to learn the value of each of the dispatch actions in the action space under the state space for compressing the waiting time of passengers in the plurality of floors, the boarding time of passengers in the plurality of elevators, and the energy consumption of the plurality of elevators as a whole; executing the dispatch action with the highest value for each of the elevators to compress the waiting time of passengers in the plurality of floors, the boarding time of passengers in the plurality of elevators, and the energy consumption of the plurality of elevators as a whole; wherein the method further comprises: during a time period between the last execution of the dispatch action and the current execution of the dispatch action, dividing a plurality of time intervals according to events unrelated to the call signal occurring in the plurality of elevators; in each of the time intervals, calculating a first index value representing the waiting time of passengers in the building; in each of the time intervals, calculating a second index value representing the boarding time of passengers in the plurality of elevators; in each of the time intervals, calculating a third index value representing the energy consumption of the plurality of elevators; fusing the first index value, the second index value, and the third index value into a target index value; the target index value is negatively correlated with the first index value, the second index value, and the third index value; during the time period, integrating the product of a preset discount factor and the target index value in time to obtain an incentive value; updating the Q-learning model of reinforcement learning according to the incentive value.

2. The method of claim 1, wherein, The method further comprises: constructing a state space in a reinforcement learning model; the state space is a matrix, the first floor identifier, the column includes the second floor identifier, the up call light time, the down call light time, and the in-call identifier, the up non-stop identifier, the down non-stop identifier, and the stop identifier of each of the elevators; querying the call signal received by each of the elevators in real time; taking the floor number indicated by the call signal as the first floor identifier, and the floor number indicated by the call signal as the second floor identifier, modifying the elements of the intersection of the first floor identifier and the second floor identifier from 0 to 1; statistically analyzing the time length of the in-call signal in the call signal; if the in-call signal is upward, taking the floor number indicated by the call signal as the first floor identifier, and the time length as the up call light time, assigning the time length to the elements of the intersection of the first floor identifier and the up call light time. If the external call signal is a down call, the floor number indicated by the call signal as the departure floor is a first floor identifier, the time length is a down call light time, and the time length is assigned to elements intersected by the first floor identifier and the down call light time; For each of the elevators, when the call signal is an internal call signal, the floor number indicated by the internal call signal as the arrival floor is a first floor identifier, and elements intersected by the first floor identifier and the internal call identifier of the elevator are modified from 0 to 1; For each of the elevators, when the elevator moves upwards, the floor number indicated by the call signal as the arrival floor is a first floor identifier, and elements intersected by the first floor identifier and the upward non-stopping identifier of the elevator are modified from 0 to 1; For each of the elevators, when the elevator moves downwards, the floor number indicated by the call signal as the arrival floor is a first floor identifier, and elements intersected by the first floor identifier and the downward non-stopping identifier of the elevator are modified from 0 to 1; For each of the elevators, the floor number indicated by the call signal as the arrival floor is a first floor identifier, and elements intersected by the first floor identifier and the stopping identifier of the elevator are modified from 0 to 1.

3. The method of claim 1, wherein, The scheduling actions include stopping at the next floor with a call signal in the moving direction, passing through the next floor with a call signal in the moving direction, and waiting at the current floor.

4. The method of claim 1, wherein, In each of the time intervals, a first index value representing the waiting time of passengers in the building is calculated, including: In each of the time intervals, the end time point of the time interval is subtracted from the time at which the passengers arrive at the hall of the floor to obtain a first interval time for which the passengers wait for the elevator; The first interval time is subjected to power operation with a preset adjustment coefficient as the index to obtain a second interval time; The second interval times corresponding to each of the passengers are summed to obtain the first index value representing the waiting time of passengers in the building; In each of the time intervals, a second index value representing the boarding time of passengers in the multiple elevators is calculated, including: In each of the time intervals, the difference between the end time point of the time interval and the time at which the passengers enter the car of the elevator is taken as a third interval time; The third interval time is subjected to power operation with a preset adjustment coefficient as the index to obtain a fourth interval time; The fourth interval times corresponding to each of the passengers are summed to obtain the second index value representing the boarding time of passengers in the multiple elevators. The adjustment coefficient is a natural number.

5. The method of claim 4, wherein, In each of the time intervals, a third index value representing the energy consumption of the multiple elevators is calculated, including: In each of the time intervals, the number of times that the multiple elevators stop at the floor is added to obtain a total stopping number; A preset energy consumption coefficient is multiplied by the total stopping number to obtain the third index value representing the energy consumption of the multiple elevators.

6. The method of claim 5, wherein, The fusing the first indicator value, the second indicator value and the third indicator value into a target indicator value comprises: adding a product of the first indicator value and a preset first weight, a product of the second indicator value and a preset second weight, and a product of the third indicator value and a preset third weight to obtain a candidate indicator value; taking the opposite of the candidate indicator value to obtain the target indicator value.

7. The method of claim 6, wherein, Further comprising: collecting distribution information of passengers in the plurality of elevators on the plurality of floors; dynamically adjusting the first weight, the second weight and the third weight according to the distribution information through a proportional-integral-derivative control algorithm; or, updating the first weight, the second weight and the third weight in a deep deterministic policy gradient framework to compress the waiting time of passengers in the plurality of floors, the boarding time of passengers in the plurality of elevators and the energy consumption of the plurality of elevators as a whole.

8. A computer device, comprising: The computer device comprises: one or more processors; a storage device for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the elevator group scheduling method as claimed in any one of claims 1-7.

9. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the elevator group scheduling method as claimed in any one of claims 1-7.

Images