Elevator dispatching method and system, electronic equipment and storage medium

Abstract

The invention discloses an elevator dispatching method and system, electronic equipment and a storage medium, and the method comprises the steps that an elevator dispatching environment is constructed according to the elevator state and calling information, and the elevator dispatching environment comprises a state space, an action space and a reward function; reinforcement learning processing is conducted on the elevator dispatching environment according to a double-Q learning model, and an initial dispatching strategy is obtained; performing global optimization processing on the initial scheduling strategy according to a simulated annealing algorithm to obtain a target scheduling strategy; and operation control is conducted on the elevator system according to the target scheduling strategy, and the double-Q learning model is updated according to an operation result. The embodiment of the invention can improve the elevator dispatching efficiency, and can be widely applied to the technical field of elevators.

Description

Technical Field

[0001] The present application relates to the technical field of elevators, and in particular to an elevator dispatching method, system, electronic equipment and storage medium. Background Art

[0002] With the acceleration of urbanization, efficient dispatching of elevator systems in high-rise buildings has become an important part of improving user experience and energy saving. In related technologies, elevator dispatching methods usually rely on rule-based strategies and cannot adapt to dynamic environments, resulting in the inability to efficiently dispatch elevators in dynamic environments, affecting the intelligence level of group elevator dispatching systems and reducing elevator dispatching efficiency. In summary, the technical problems existing in related technologies need to be improved. Summary of the invention

[0003] The main purpose of the embodiments of the present application is to provide an elevator dispatching method, system, electronic device and storage medium, which can improve the efficiency of elevator dispatching.

[0004] To achieve the above object, one aspect of an embodiment of the present application provides an elevator dispatching method, the method comprising:

[0005] An elevator dispatching environment is constructed according to the elevator state and the call information, wherein the elevator dispatching environment includes a state space, an action space and a reward function;

[0006] Performing reinforcement learning processing on the elevator dispatching environment according to the double Q learning model to obtain an initial dispatching strategy;

[0007] Performing global optimization processing on the initial scheduling strategy according to a simulated annealing algorithm to obtain a target scheduling strategy;

[0008] The elevator system is operated and controlled according to the target scheduling strategy, and the double-Q learning model is updated according to the operation result.

[0009] In some embodiments, the step of constructing an elevator dispatching environment according to the elevator state and the call information includes the following steps:

[0010] The state space is constructed according to the elevator state and the call information, wherein the elevator state includes the current floor, the running direction and the load state of each elevator in the elevator system, and the call information includes the current floor, the target floor and the waiting time;

[0011] Selecting an elevator from the elevator system to respond according to the call information, and constructing the action space according to the response action;

[0012] The reward function is constructed according to the waiting time and the elevator running path.

[0013] In some embodiments, performing reinforcement learning processing on the elevator dispatching environment according to the double Q learning model to obtain an initial dispatching strategy includes the following steps:

[0014] The double Q learning model is constructed, and the double Q learning model includes a principal value network and a target value network;

[0015] Acquire the current state from the elevator dispatching environment, and alternately select the principal value network or the target value network from the dual Q learning model to obtain the current network;

[0016] The current state is input into the current network to obtain the initial scheduling strategy.

[0017] In some embodiments, performing global optimization processing on the initial scheduling strategy according to a simulated annealing algorithm to obtain a target scheduling strategy includes the following steps:

[0018] Performing disturbance update processing on the initial scheduling strategy according to the simulated annealing algorithm to obtain candidate actions;

[0019] The dual-Q learning model is subjected to a strategy update process according to the candidate actions to obtain the target scheduling strategy.

[0020] In some embodiments, performing disturbance update processing on the initial scheduling strategy according to the simulated annealing algorithm to obtain candidate actions includes the following steps:

[0021] Determining an action selection function according to the simulated annealing algorithm;

[0022] The initial scheduling strategy is subjected to action selection processing according to the action selection function to obtain the candidate action.

[0023] In some embodiments, performing a policy update process on the dual-Q learning model according to the candidate action to obtain the target scheduling policy includes the following steps:

[0024] Inputting the candidate action into the dual-Q learning model for updating to obtain an updated initial scheduling strategy;

[0025] The temperature parameter is updated, and the process returns to the simulated annealing algorithm to perform disturbance update processing on the initial scheduling strategy until the temperature parameter reaches a convergence condition, thereby obtaining the target scheduling strategy.

[0026] In some embodiments, updating the dual-Q learning model according to the operation result includes the following steps:

[0027] Performing index collection and processing on the elevator system according to the operation results to obtain performance indicators;

[0028] Performing evaluation processing on the performance indicator to obtain an evaluation result;

[0029] The reward function is updated according to the evaluation result, and the parameters of the dual-Q learning model are updated according to the updated reward function.

[0030] To achieve the above object, another aspect of the embodiment of the present application provides an elevator dispatching system, the elevator dispatching system comprising:

[0031] The first module is used to construct an elevator dispatching environment according to the elevator state and the user request, wherein the elevator dispatching environment includes a state space, an action space and a reward function;

[0032] The second module is used to perform reinforcement learning processing on the elevator scheduling environment according to the double Q learning model to obtain an initial scheduling strategy;

[0033] The third module is used to perform global optimization processing on the initial scheduling strategy according to a simulated annealing algorithm to obtain a target scheduling strategy;

[0034] The fourth module is used to control the operation of the elevator system according to the target scheduling strategy, and update the double Q learning model according to the operation results.

[0035] To achieve the above-mentioned purpose, another aspect of an embodiment of the present application provides an electronic device, the electronic device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the above-mentioned method when executing the computer program.

[0036] To achieve the above objective, another aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described above is implemented.

[0037] The embodiments of the present application include at least the following beneficial effects: the present application provides an elevator scheduling method, system, electronic device and storage medium, the scheme constructs an elevator scheduling environment according to the elevator state and call information, and performs reinforcement learning processing on the elevator scheduling environment according to the double Q learning model to obtain an initial scheduling strategy, which can reduce the overestimation problem through the double Q learning model; and, the scheme performs global optimization processing on the initial scheduling strategy according to the simulated annealing algorithm to obtain a target scheduling strategy, and can introduce a simulated annealing algorithm to perform global optimization on the current strategy, and dynamically adjust the step size and temperature parameters to reduce the problem of the current strategy falling into the local optimum; in addition, the scheme controls the operation of the elevator system according to the target scheduling strategy, and updates the double Q learning model according to the operation results, which can adjust the scheduling strategy in real time, while optimizing the operation efficiency and energy consumption. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 is a flow chart of an elevator dispatching method provided by an embodiment of the present application;

[0039] Figure 2 This is an example diagram of elevator operation using the nearest principle scheduling scheme to simulate peak passenger flow;

[0040] Figure 3 This is an example diagram of elevator operation using the nearest principle scheduling scheme to simulate the morning peak passenger flow;

[0041] Figure 4 This is an example diagram of elevator operation for simulating peak passenger flow using the solution of an embodiment of the present application;

[0042] Figure 5 This is an example diagram of elevator operation for simulating passenger flow during the morning rush hour using the solution of an embodiment of the present application;

[0043] Figure 6 It is a comparison chart of the average waiting time of passengers in the two schemes;

[0044] Figure 7 It is a comparison chart of the total path length of the two schemes;

[0045] Figure 8 It is a structural schematic diagram of an elevator dispatching system provided in an embodiment of the present application;

[0046] Fig. 9 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below in conjunction with the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present application. They are only examples of systems and methods consistent with some aspects of the embodiments of the present application as detailed in the attached claims.

[0048] It is understood that the terms "first", "second", etc. used in this application can be used to describe various concepts in this article, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another concept. For example, without departing from the scope of the embodiment of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" and "if" as used herein can be interpreted as "at the time of" or "when" or "in response to determination".

[0049] The terms "at least one", "multiple", "each", "any", etc. used in this application, at least one includes one, two or more, multiple includes two or more, each refers to each of the corresponding multiple, and any refers to any one of the multiple.

[0050] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0051] Before describing the embodiments of the present application in detail, some nouns and terms involved in the embodiments of the present application are first described. The nouns and terms involved in the embodiments of the present application are subject to the following explanations.

[0052] Reinforcement Learning (RL) is a machine learning method. The basic framework of reinforcement learning is the Markov decision process, which allows the agent to learn the optimal strategy through trial and error in the interaction with the environment. The agent performs actions in the environment and receives feedback, i.e. rewards, based on the results of the actions. These reward signals guide the agent to adjust its strategy to maximize the long-term cumulative rewards.

[0053] With the acceleration of urbanization, efficient scheduling of elevator systems in high-rise buildings has become an important part of improving user experience and energy conservation. In related technologies, elevator scheduling methods usually rely on rule-based strategies and cannot adapt to dynamic environments. In recent years, reinforcement learning and intelligent optimization algorithms have shown significant advantages in complex scheduling problems. However, a single reinforcement learning algorithm may face slow convergence or local optimality problems, and traditional optimization algorithms also have the problem of difficulty in balancing global search and local convergence in multi-objective problems.

[0054] In view of this, an elevator scheduling method, system, electronic device and storage medium are provided in the embodiments of the present application. The scheme constructs an elevator scheduling environment according to the elevator state and call information, and performs reinforcement learning processing on the elevator scheduling environment according to the double Q learning model to obtain an initial scheduling strategy, which can reduce the overestimation problem through the double Q learning model; and, the scheme performs global optimization processing on the initial scheduling strategy according to the simulated annealing algorithm to obtain a target scheduling strategy, and can introduce a simulated annealing algorithm to perform global optimization on the current strategy, and dynamically adjust the step size and temperature parameters to reduce the problem of the current strategy falling into a local optimum; in addition, the scheme controls the operation of the elevator system according to the target scheduling strategy, and updates the double Q learning model according to the operation results, which can adjust the scheduling strategy in real time, while optimizing the operation efficiency and energy consumption.

[0055] An elevator dispatching method provided in an embodiment of the present application relates to the field of elevator technology. The elevator dispatching method provided in an embodiment of the present application can be applied to a control terminal, can be applied to a server, or can be software running in a control terminal or a server. In some embodiments, the control terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc., but is not limited to this; the server side can be configured as an independent physical server, or can be configured as a server cluster or a distributed system composed of multiple physical servers, and can also be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network; the software can be an application that implements the elevator dispatching method, etc., but is not limited to the above forms.

[0056] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0057] Figure 1is an optional flow chart of an elevator dispatching method provided in an embodiment of the present application. Figure 1 The method may include but is not limited to steps S101 to S104.

[0058] Step S101, constructing an elevator dispatching environment according to the elevator state and call information, wherein the elevator dispatching environment includes a state space, an action space and a reward function;

[0059] Step S102, performing reinforcement learning processing on the elevator dispatching environment according to the double Q learning model to obtain an initial dispatching strategy;

[0060] Step S103, performing global optimization processing on the initial scheduling strategy according to a simulated annealing algorithm to obtain a target scheduling strategy;

[0061] Step S104, controlling the operation of the elevator system according to the target scheduling strategy, and updating the double Q learning model according to the operation result.

[0062] Steps S101 to S104 shown in the embodiment of the present application can realize fast, intelligent, and globally optimized elevator scheduling in a dynamic environment by combining the double Q learning model and the simulated annealing algorithm to schedule the elevator system. Specifically, an elevator scheduling environment is constructed based on the elevator state and call information, and the elevator scheduling environment is used as an environment for reinforcement learning. Then, the elevator scheduling environment is reinforced learned according to the double Q learning model, and an initial scheduling strategy is generated through the model. The double Q learning model obtained by training in the embodiment of the present application can reduce the overestimation problem in single Q learning, and can make full use of historical data for training with the help of a distributed training mechanism, thereby improving the stability of the algorithm. Then, the initial scheduling strategy is globally optimized according to the simulated annealing algorithm to obtain the target scheduling strategy, and the simulated annealing algorithm can be introduced to globally optimize the current strategy, thereby dynamically adjusting the step size and temperature parameters in the exploration-utilization trade-off process to avoid falling into the problem of local optimality. Finally, the elevator system is operated and controlled according to the target scheduling strategy, and the double Q learning model is updated according to the operation results, which can weigh multiple objectives, optimize operating efficiency and energy consumption at the same time, and generate elevator scheduling decisions in real time according to passenger requests.

[0063] In step S101 of some embodiments, constructing an elevator dispatching environment according to the elevator state and call information includes the following steps:

[0064] The state space is constructed according to the elevator state and the call information, wherein the elevator state includes the current floor, the running direction and the load state of each elevator in the elevator system, and the call information includes the current floor, the target floor and the waiting time;

[0065] Selecting an elevator from the elevator system to respond according to the call information, and constructing the action space according to the response action;

[0066] The reward function is constructed according to the waiting time and the elevator running path.

[0067] In the embodiment of the present application, the elevator dispatching environment is an environment for reinforcement learning, which can be constructed based on the elevator state and call information. The elevator state includes the current floor, running direction and load status of each elevator in the elevator system. The call information is the information of the user requesting the elevator, which may include the current floor, target floor and waiting time. The current floor is the floor where the user initiates the request, the target floor is the floor that the user needs to reach, the current floor and the target floor can be obtained by interacting with the elevator system, and the waiting time is the time the user waits for the elevator to arrive. The state space is obtained by constructing the elevator state and call information, that is, the state space includes the floor where the elevator is located, the running direction, the load status and the unfinished passenger call information. According to the call information, an elevator is selected from the elevator system to respond, that is, an elevator is selected from multiple elevators in the elevator system to respond to the current user's call. The response action is the action of selecting the elevator, and the action space is constructed based on the response action. The reward function is constructed by the waiting time and the elevator running path, and the comprehensive optimization target is defined by the reward function. The comprehensive optimization target is used to consider the user's waiting time and the elevator's operating efficiency and energy consumption. The formula of the reward function is as follows:

[0068]

[0069] In the formula, c t is the total waiting time of passengers at time t, β is the exponential decay rate, and p(s,a) is the distance that the elevator selected by action a needs to travel in response to the unfinished passenger call information in state s.

[0070] One of the above technical solutions has the following advantages or beneficial effects: the embodiment of the present application constructs an elevator dispatching environment, and can perform reinforcement learning processing based on the elevator dispatching environment, thereby providing a basis for the subsequent generation of elevator dispatching strategies.

[0071] In some embodiments, performing reinforcement learning processing on the elevator dispatching environment according to the double Q learning model to obtain an initial dispatching strategy includes the following steps:

[0072] The double Q learning model is constructed, and the double Q learning model includes a principal value network and a target value network;

[0073] Acquire the current state from the elevator dispatching environment, and alternately select the principal value network or the target value network from the dual Q learning model to obtain the current network;

[0074] The current state is input into the current network to obtain the initial scheduling strategy.

[0075] In the embodiment of the present application, a dual Q learning model is constructed, and the principal value network and the target value network of the dual Q learning model are initialized, and the model parameters such as the learning rate and the discount factor are set, wherein the principal value network and the target value network have the same structure, and are used to calculate the reward value of the elevator scheduling strategy respectively, so as to avoid overestimation. Then the current state is obtained from the elevator scheduling environment, and the current state can be obtained by real-time acquisition of the elevator state and call information, which can be obtained by the sensor module and the scheduling system, or the current position, running direction and user call distribution of the elevator can be obtained in real time through the elevator group control simulation system. The obtained data is converted into the current state and input into the dual Q learning model, and then the principal value network or the target value network is alternately selected as the current network in the dual Q learning model, and the current state is input into the current network for action selection to obtain the initial scheduling strategy, which includes the current state, the current action and the current action value. It should be noted that the current network selected in the embodiment of the present application is used to select the optimal action, and at this time another network is used to evaluate the value of the action.

[0076] One of the above technical solutions has the following advantages or beneficial effects: The embodiment of the present application introduces two independent Q functions through a dual Q learning model to separate the action selection and action value update processes, which helps to reduce the deviation caused by overestimation of the optimal action value and can improve the stability of the strategy.

[0077] In some embodiments, performing global optimization processing on the initial scheduling strategy according to a simulated annealing algorithm to obtain a target scheduling strategy includes the following steps:

[0078] Performing disturbance update processing on the initial scheduling strategy according to the simulated annealing algorithm to obtain candidate actions;

[0079] The dual-Q learning model is subjected to a strategy update process according to the candidate actions to obtain the target scheduling strategy.

[0080] In the embodiment of the present application, the initial scheduling strategy generated by the dual Q learning model is used as input, and the simulated annealing algorithm is used for global optimization. The initial scheduling strategy is perturbed by the simulated annealing algorithm to generate a new candidate solution, and the action selection function can be used to determine whether to generate a new candidate action. The principal value network and the target value network are updated by inputting the candidate action into the dual Q learning model, and the final target scheduling strategy is output.

[0081] One of the above technical solutions has the following advantages or beneficial effects: the embodiment of the present application introduces a simulated annealing algorithm to globally optimize the current strategy, which can balance the problems of global search and local convergence and improve the scheduling efficiency of the elevator scheduling strategy.

[0082] In some embodiments, performing disturbance update processing on the initial scheduling strategy according to the simulated annealing algorithm to obtain candidate actions includes the following steps:

[0083] Determining an action selection function according to the simulated annealing algorithm;

[0084] The initial scheduling strategy is subjected to action selection processing according to the action selection function to obtain the candidate action.

[0085] In the embodiment of the present application, the current scheduling scheme is disturbed according to the simulated annealing algorithm to generate a new candidate scheme, and the initial scheduling strategy can be selected by the action selection function to obtain the candidate action, wherein the action selection function is shown as follows:

[0086]

[0087] In the formula, a n is the nth action that can be taken in the current state, n is a positive integer, x is a random number in the interval [0,1], Q(s,a n ) represents the function value obtained by the current network evaluating the actions that can be taken, and T represents the temperature parameter.

[0088] One of the above technical solutions has the following advantages or beneficial effects: the embodiment of the present application selects actions for the initial scheduling strategy through an action selection function, and can update the current action, thereby adjusting the step size and temperature parameters during the exploration process of the model to avoid falling into local optimality.

[0089] In some embodiments, performing a policy update process on the dual-Q learning model according to the candidate action to obtain the target scheduling policy includes the following steps:

[0090] Inputting the candidate action into the dual-Q learning model for updating to obtain an updated initial scheduling strategy;

[0091] The temperature parameter is updated, and the process returns to the simulated annealing algorithm to perform disturbance update processing on the initial scheduling strategy until the temperature parameter reaches a convergence condition, thereby obtaining the target scheduling strategy.

[0092] In the embodiment of the present application, the candidate actions selected by the simulated annealing algorithm are input into the double Q learning model for update processing, and the update formula is as follows:

[0093] Q1(s t ,a t )←(1-α)Q1(s t ,a t )+α[R(s t ,a t )+γQ2(s t+1 ,f SA (Q1,s t ))];

[0094] Q2(s t ,a t )←(1-α)Q2(s t ,a t )+α[R(s t ,a t )+γQ1(s t+1 ,f SA (Q2,s t ))];

[0095] In the formula, Q1(s t ,a t ) represents the function value of the principal value network, Q2(s t ,a t ) represents the function value of the target value network, α represents the learning rate, γ represents the discount factor, R(s t ,a t ) represents the reward function, s t represents the state at time t, a t represents the action at time t, f Sa Represents a candidate action.

[0096] At this time, a new initial scheduling strategy is generated through the updated double Q learning model, and the temperature parameters are updated, returning to the step of perturbing and updating the initial scheduling strategy according to the simulated annealing algorithm, and the cycle is performed until the temperature parameters reach the convergence condition. The temperature parameters are the initial temperature T0 and annealing rate η determined by the simulated annealing algorithm, according to T = T0*η k , gradually reduce the temperature T until convergence, and output the final target scheduling strategy.

[0097] In some embodiments, updating the dual-Q learning model according to the operation result includes the following steps:

[0098] Performing index collection and processing on the elevator system according to the operation results to obtain performance indicators;

[0099] Performing evaluation processing on the performance indicator to obtain an evaluation result;

[0100] The reward function is updated according to the evaluation result, and the parameters of the dual-Q learning model are updated according to the updated reward function.

[0101] In an embodiment of the present application, the optimized target scheduling strategy is input into the elevator system, and the elevator system controls the elevator to operate according to the target scheduling strategy, which can dynamically adjust the operating state of the elevator to ensure real-time response to new passenger requests. After running the target scheduling strategy, key performance indicators of the elevator system are collected according to the operating results of the elevator system to obtain performance indicators, which may include average waiting time, total path length, energy consumption, etc. The performance indicators are then evaluated and processed, and performance evaluation can be performed through neural networks, etc. to obtain evaluation results. The reward function is then updated according to the evaluation results, thereby updating the parameters of the dual Q learning model according to the updated reward function, and further optimizing the dual Q learning model.

[0102] One of the above technical solutions has the following advantages or beneficial effects: the embodiment of the present application can realize efficient scheduling of elevators in a dynamic environment, taking into account passenger waiting time, operating efficiency and energy consumption, and significantly improving the intelligence level of the group elevator scheduling system.

[0103] The following is a detailed description of the embodiments of the present invention with reference to specific application examples:

[0104] The embodiments of the present application can be widely used in the field of elevator technology, and simulation experiments are used to verify the scheduling performance of the method of the embodiments of the present application in a dynamic environment to adapt to different load conditions. Among them, the experimental platform is a simulation platform based on MATLAB, and the elevator system is configured with 6 elevators, serving a virtual commercial building with 30 floors. The experimental environment simulates two different load scenarios, namely morning peak and off-peak, by randomly generating passenger call data. The algorithm is configured as a principal value network and a target network with the same structure for double Q learning, the learning rate is set to 0.01, the exponential decay rate is 0.01, the simulated annealing initial temperature is 100, and the annealing rate is 0.9. The data collected is dynamically generated call data including floor distribution, arrival time and load status. The experimental process is as follows: initialize the initial position and state of the elevator, and randomly generate passenger call distribution in different time periods. Run the double Q learning algorithm to generate a scheduling strategy, and perform simulated annealing optimization. Simulate the elevator scheduling process and record the average waiting time, response time and operating efficiency of each floor. The common elevator group control scheduling scheme, namely the nearest principle, is used to simulate passenger flow. The elevator operation results are as follows: Figure 2 and Figure 3 Then the scheduling scheme obtained by double Q learning and simulated annealing in the embodiment of the present application is used to simulate passenger flow, and the elevator operation results are shown in Figure 4 and Figure 5 Then, the scheduling performance in different scenarios is compared and analyzed. The comparison results are shown in Figure 6 and Figure 7 As shown, in the morning peak scenario (entry passenger flow): the average waiting time of passengers in the scheduling scheme obtained by double Q learning and simulated annealing is 44.84 seconds, which is 29.7% shorter than the average waiting time of 63.86 seconds of passengers in the traditional scheduling algorithm based on the proximity rule; the total elevator operation path is reduced by 7.8%, and the program running time is 2.607 seconds. In the flat peak scenario (inter-layer passenger flow): the average waiting time of passengers in the scheduling scheme obtained by double Q learning and simulated annealing is 33.88 seconds, which is 20% shorter than the average waiting time of 42.35 seconds of passengers in the traditional scheduling algorithm based on the proximity rule; the total elevator operation path is reduced by 15.3%, and the program running time is 3.155 seconds. Therefore, the embodiment method of the present application has good dynamic adaptability and robustness, and can effectively improve the scheduling efficiency and energy saving effect of the elevator system under different load scenarios.

[0105] In summary, the elevator dispatching method proposed in the embodiment of the present application can generate an optimized dispatching scheme in real time and efficiently in a complex elevator operation environment. The stability of the strategy is improved by the dual Q learning model, and the simulated annealing algorithm is combined for global optimization, which effectively shortens the waiting time of passengers and reduces the total operation path and energy consumption of the elevator. The embodiment of the present application shows high applicability and robustness in both actual scenarios and simulation conditions, and is particularly suitable for application in the dispatching management of multi-elevator systems such as large commercial buildings and office buildings.

[0106] See also Figure 8 The embodiment of the present application also provides an elevator dispatching system, which can implement the above elevator dispatching method, and the system includes:

[0107] The first module 801 is used to construct an elevator dispatching environment according to the elevator state and the user request, wherein the elevator dispatching environment includes a state space, an action space and a reward function;

[0108] The second module 802 is used to perform reinforcement learning processing on the elevator scheduling environment according to the double Q learning model to obtain an initial scheduling strategy;

[0109] The third module 803 is used to perform global optimization processing on the initial scheduling strategy according to a simulated annealing algorithm to obtain a target scheduling strategy;

[0110] The fourth module 804 is used to control the operation of the elevator system according to the target scheduling strategy, and update the double Q learning model according to the operation result.

[0111] It can be understood that the contents of the above method embodiments are all applicable to the present system embodiments, the functions specifically implemented by the present system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0112] The embodiment of the present application also provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the above elevator scheduling method when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0113] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0114] See also Fig. 9 , Fig. 9 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0115] The processor 901 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0116] The memory 902 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 902 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 902, and the processor 901 calls and executes the elevator dispatching method of the embodiment of this application;

[0117] Input / output interface 903, used to implement information input and output;

[0118] Communication interface 904, used to realize communication interaction between the device and other devices, which can be realized by wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WIFI, Bluetooth, etc.);

[0119] A bus 905 that transmits information between various components of the device (e.g., the processor 901, the memory 902, the input / output interface 903, and the communication interface 904);

[0120] The processor 901 , the memory 902 , the input / output interface 903 and the communication interface 904 are connected to each other in communication within the device via a bus 905 .

[0121] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the above-mentioned elevator scheduling method is implemented.

[0122] It can be understood that the contents of the above method embodiments are all applicable to the present storage medium embodiments, the functions specifically implemented by the present storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0123] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0124] The embodiments of the present application provide an elevator dispatching method, system, electronic device and storage medium. The scheme constructs an elevator dispatching environment according to the elevator status and call information, performs reinforcement learning processing on the elevator dispatching environment according to the double Q learning model to obtain an initial dispatching strategy, and can reduce the overestimation problem through the double Q learning model; and the scheme performs global optimization processing on the initial dispatching strategy according to the simulated annealing algorithm to obtain a target dispatching strategy, and can introduce a simulated annealing algorithm to perform global optimization on the current strategy, and dynamically adjust the step size and temperature parameters to reduce the problem of the current strategy falling into the local optimum; in addition, the scheme controls the operation of the elevator system according to the target dispatching strategy, and updates the double Q learning model according to the operation results, so as to adjust the dispatching strategy in real time and optimize the operation efficiency and energy consumption at the same time.

[0125] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0126] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0127] The system embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment.

[0128] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0129] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0130] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0131] In the several embodiments provided in the present application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of systems or units, which can be electrical, mechanical or other forms.

[0132] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0133] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0134] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0135] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. An elevator dispatching method, characterized in that: The method comprises the following steps: An elevator dispatching environment is constructed according to the elevator state and the call information, wherein the elevator dispatching environment includes a state space, an action space and a reward function; Performing reinforcement learning processing on the elevator dispatching environment according to the double Q learning model to obtain an initial dispatching strategy; Performing global optimization processing on the initial scheduling strategy according to a simulated annealing algorithm to obtain a target scheduling strategy; The elevator system is operated and controlled according to the target scheduling strategy, and the double-Q learning model is updated according to the operation result.

2. The method according to claim 1, characterized in that The method of constructing an elevator dispatching environment according to the elevator state and the call information includes the following steps: The state space is constructed according to the elevator state and the call information, wherein the elevator state includes the current floor, the running direction and the load state of each elevator in the elevator system, and the call information includes the current floor, the target floor and the waiting time; Selecting an elevator from the elevator system to respond according to the call information, and constructing the action space according to the response action; The reward function is constructed according to the waiting time and the elevator running path.

3. The method according to claim 1, characterized in that The step of performing reinforcement learning processing on the elevator dispatching environment according to the double Q learning model to obtain an initial dispatching strategy includes the following steps: The double Q learning model is constructed, and the double Q learning model includes a principal value network and a target value network; Acquire the current state from the elevator dispatching environment, and alternately select the principal value network or the target value network from the dual Q learning model to obtain the current network; The current state is input into the current network to obtain the initial scheduling strategy.

4. The method according to claim 1, characterized in that: The method of performing global optimization processing on the initial scheduling strategy according to the simulated annealing algorithm to obtain the target scheduling strategy includes the following steps: Performing disturbance update processing on the initial scheduling strategy according to the simulated annealing algorithm to obtain candidate actions; The dual-Q learning model is subjected to a strategy update process according to the candidate actions to obtain the target scheduling strategy.

5. The method according to claim 4, characterized in that The method of performing disturbance update processing on the initial scheduling strategy according to the simulated annealing algorithm to obtain candidate actions includes the following steps: Determining an action selection function according to the simulated annealing algorithm; The initial scheduling strategy is subjected to action selection processing according to the action selection function to obtain the candidate action.

6. The method according to claim 4, characterized in that The step of performing a strategy update process on the dual-Q learning model according to the candidate action to obtain the target scheduling strategy includes the following steps: Inputting the candidate action into the dual-Q learning model for updating to obtain an updated initial scheduling strategy; The temperature parameter is updated, and the process returns to the simulated annealing algorithm to perform disturbance update processing on the initial scheduling strategy until the temperature parameter reaches a convergence condition, thereby obtaining the target scheduling strategy.

7. The method according to any one of claims 1 to 6, characterized in that: The updating of the dual-Q learning model according to the operation result comprises the following steps: Performing index collection and processing on the elevator system according to the operation results to obtain performance indicators; Performing evaluation processing on the performance indicator to obtain an evaluation result; The reward function is updated according to the evaluation result, and the parameters of the dual-Q learning model are updated according to the updated reward function.

8. An elevator dispatching system, characterized in that: The elevator dispatching system comprises: The first module is used to construct an elevator dispatching environment according to the elevator state and the user request, wherein the elevator dispatching environment includes a state space, an action space and a reward function; The second module is used to perform reinforcement learning processing on the elevator scheduling environment according to the double Q learning model to obtain an initial scheduling strategy; The third module is used to perform global optimization processing on the initial scheduling strategy according to a simulated annealing algorithm to obtain a target scheduling strategy; The fourth module is used to control the operation of the elevator system according to the target scheduling strategy, and update the double Q learning model according to the operation results.

9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images