Automatic task allocation method and system for elevator emergency rescue platform

Abstract

The invention provides an automatic task allocation method and system for an elevator emergency rescue platform, and relates to the technical field of artificial intelligence, and the method comprises the steps: obtaining elevator operation state data and environment information data, constructing an emergency task model, and obtaining classified emergency tasks; inputting the classified emergency tasks into a deep reinforcement learning model to generate a task allocation scheme; the deep reinforcement learning model is formed by combining a deep Q network structure with a priority scanning algorithm, and a priority experience playback mechanism is introduced; and constructing a multi-objective optimization module comprising a time constraint, a space constraint and a resource constraint, and optimizing the task allocation scheme by the multi-objective optimization module based on an improved particle swarm algorithm to obtain an optimal task allocation scheme.

Description

Technical Field

[0001] The present invention relates to artificial intelligence technology, and in particular to an automated task allocation method and system for an elevator emergency rescue platform. Background Art

[0002] Elevator emergency rescue is a complex and urgent task, involving coordination and resource allocation in multiple aspects. Traditional elevator emergency rescue task allocation mainly relies on manual experience, which has the following problems: Low decision-making efficiency: In emergency situations, manual decision-making often has difficulty processing large amounts of information quickly, leading to rescue delays.

[0003] Improper resource allocation: Due to the lack of global optimization, some resources may be overused while other resources are idle.

[0004] Difficulty in dealing with complex scenarios: When multiple elevators fail at the same time or multiple emergency situations occur simultaneously, it is difficult for humans to quickly develop the best solution.

[0005] Lack of data support: Traditional methods cannot fully utilize historical and real-time data, and cannot conduct accurate risk assessment and success rate prediction.

[0006] Difficulty in making multi-objective trade-offs: Rescue missions usually involve multiple objectives such as time, safety, and resource utilization. It is difficult for humans to find the best balance in a short period of time.

[0007] Lack of adaptive capabilities: Unable to quickly adjust the task allocation plan according to dynamic changes in the rescue process.

[0008] Low standardization: Different rescuers may adopt different decision-making criteria, resulting in unstable rescue quality. Summary of the invention

[0009] The present invention provides an automated task allocation method and system for an elevator emergency rescue platform, which can solve the problems in the prior art.

[0010] A first aspect of the present invention, Provides an automated task allocation method for an elevator emergency rescue platform, including: Acquire elevator operation status data and environmental information data, perform data cleaning and standardization on the elevator operation status data and environmental information data to obtain preprocessed data; construct an emergency task model based on the preprocessed data, classify the emergency task model, and obtain classified emergency tasks; The classified emergency tasks are input into the deep reinforcement learning model to generate a task allocation plan; the deep reinforcement learning model is composed of a deep Q network structure combined with a priority scanning algorithm, and a priority experience replay mechanism is introduced; the state space of the deep reinforcement learning model includes currently available rescue resources, task urgency and geographic location information, and the action space includes rescue personnel selection and path planning decisions; based on the state space and action space, a reward function is designed, and the reward function takes into account task completion time, resource utilization and rescue success rate; A multi-objective optimization module including time constraints, space constraints and resource constraints is constructed. The multi-objective optimization module optimizes the task allocation scheme based on the improved particle swarm algorithm to obtain the optimal task allocation scheme; based on the optimal task allocation scheme, a task execution scheme is generated; the optimal task allocation scheme is sent to the real-time monitoring module, and the real-time monitoring module monitors the task execution process and collects feedback information; the feedback information is input into the deep reinforcement learning model for online learning and model updating to achieve continuous optimization of the task allocation scheme.

[0011] Preferably, The steps of constructing an emergency task model based on the preprocessed data and classifying the emergency task model to obtain the classified emergency tasks include: Receiving pre-processed elevator operation status data and environmental information data, including time series data, image data and audio data; Using a long short-term memory network to extract features from time series data, the long short-term memory network includes an input layer, two long short-term memory layers and a fully connected layer, wherein the two long short-term memory layers have different numbers of units, and each long short-term memory layer is set with dropout; Using a pre-trained residual neural network to extract features from image data, the residual neural network removes the original fully connected layer and adds a global average pooling layer and a new fully connected layer; Convert the audio data into a Mel-spectrogram, and use a convolutional neural network to extract features from the Mel-spectrogram, wherein the convolutional neural network includes two convolutional layers, two maximum pooling layers, and one fully connected layer; Apply the self-attention mechanism to the features of time series data, image data, and audio data, calculate the attention scores and obtain weighted features; The weighted features are concatenated and fused through a multi-layer perceptron to obtain fused features; Based on the fusion features, a hierarchical attention network is constructed, wherein the hierarchical attention network includes an urgency classification layer, a fault type classification layer, and a specific problem classification layer; Apply the attention mechanism in each classification layer, calculate the attention weight and obtain the weighted context vector; Use the softmax classifier to perform multi-category classification on the weighted context vector of each classification layer; A multi-task learning framework is used to combine the cross entropy loss of each classification layer to construct a comprehensive loss function; Training the hierarchical attention network using an optimizer and applying an early stopping strategy and a regularization method during the training process; The classification results of the trained hierarchical attention network are post-processed, and the post-processed classification results are output to obtain the classified emergency tasks.

[0012] Preferably, The deep reinforcement learning model is composed of a deep Q network structure combined with a priority scanning algorithm, and the steps of introducing a priority experience replay mechanism include: Constructing a deep Q network, wherein the deep Q network includes an input layer, multiple fully connected layers, an attention layer, and an output layer; establishing a main network and a target network, wherein the main network is used to select actions, and the target network is used to calculate target Q values, and periodically updating the target network parameters by soft updating; Design a priority scanning algorithm, divide the action space into layers, and design a scoring function for each action layer; The deep Q network is combined with the priority scanning algorithm, including: using the deep Q network to forward propagate the current state to obtain a preliminary Q value estimate; using the priority scanning algorithm to calculate the action priority score layer by layer starting from the top layer, where the scoring function comprehensively considers the preliminary Q value estimate and the action characteristics; selecting the top-k highest-scoring actions at each layer, and the score is the weighted sum of the preliminary Q value estimate and the action priority score; at the bottom layer, selecting the action with the highest comprehensive score from the remaining candidate actions as the final decision; Introduce a priority experience replay mechanism, use the SumTree data structure to store experience tuples and corresponding priorities, calculate the priority based on the TD error, and determine the sampling probability based on the priority; calculate the importance sampling weight; sample training data from the experience pool to train the deep reinforcement learning model; The ε-greedy strategy is used for motion exploration, and the ε value is dynamically adjusted during training; the adaptive Huber loss function is used in combination with the importance sampling weights to calculate the overall loss; the network parameters are updated through the Adam optimizer; the gradient clipping technique is applied during training to limit the L2 norm of the gradient; a batch normalization layer is added after each fully connected layer; the cosine annealing strategy is used to dynamically adjust the learning rate; the priority of the sampling experience is updated according to the new TD error obtained during training; According to the deep Q network structure, priority scanning algorithm and priority experience replay mechanism, the deep reinforcement learning model is iteratively trained until the deep reinforcement learning model converges or reaches the preset number of training times.

[0013] Preferably, The ε-greedy strategy is used for action exploration. The steps of dynamically adjusting the ε value during training include: Initialize the basic ε-greedy strategy, define the state space, action space and state-action value function; An exponential decay mechanism is introduced to dynamically adjust the ε value according to the number of training steps, so that the ε value gradually decreases from the initial high exploration rate to a stable low exploration rate; Establish a performance-driven adaptive adjustment mechanism to increase exploration when learning performance improves and reduce exploration when learning performance decreases; Combining the Thompson sampling idea, a Bayesian neural network is used to estimate the Q value distribution, and the ε value is dynamically adjusted according to the uncertainty of the Q value; A hierarchical ε-greedy strategy is constructed to adopt different exploration rates at different decision levels according to the hierarchical nature of the elevator rescue task. Dynamically calculate and update the ε value in each decision cycle; select exploration or exploitation behavior in the current state based on the updated ε value; execute the selected behavior, observe environmental feedback, and update the state-action value function; based on the new state-action value function and environmental feedback, adjust the mean and standard deviation of the Q value estimate.

[0014] Preferably, The steps to calculate the overall loss using the adaptive Huber loss function combined with the importance sampling weights include: The formula for the overall loss function is: ; in: ; ; ; Among them: Total_Loss represents the overall loss function; β j represents the weight coefficient of the jth target; w i represents the importance sampling weight of the i-th sample; L(δ i ,α(t)) represents the adaptive Huber loss function;δ i The parameter represents the prediction error of the i-th sample; α(t) represents the adaptive threshold; α base represents the basic threshold; γ represents the uncertainty adjustment coefficient; std(TD error (t)) represents the standard deviation of the TD error of the current batch; w max represents the upper limit of the importance sampling weight; p i represents the sampling probability of the i-th sample in the priority experience playback; λ represents the weight coefficient of the task urgency; E irepresents the urgency of the task related to the i-th sample; μ represents the weight coefficient of the sample novelty; N i represents the novelty score of the i-th sample; max(w) represents the maximum value of the importance sampling weights of all samples; i represents the index of the sample, and j represents the index of the target or task.

[0015] Preferably, Based on the state space and the action space, a reward function is designed, wherein the reward function considers task completion time, resource utilization rate and rescue success rate, and the steps include: Construct a reward function that combines the task completion time reward, resource utilization reward, and rescue success rate reward in a weighted manner; An exponential decay function based on time pressure is used to calculate the task completion time reward, and the reward value is dynamically adjusted according to the estimated task completion time; The resource utilization rate reward is calculated by combining the utilization rate of rescuers and equipment, and the overall resource utilization efficiency is calculated by arithmetic average; The sigmoid function based on risk assessment is used to calculate the rescue success rate reward, and the reward sensitivity is dynamically adjusted according to the difference between the estimated rescue success probability and the preset threshold; An adaptive weight mechanism is introduced to automatically adjust the weights of each component in the reward function according to the real-time change rate of each indicator during the rescue process; Use Pareto optimization method to solve the reward function and automatically adjust the trade-offs between different objectives; The reward distribution for each action is estimated through the Monte Carlo sampling method, and the mean and variance of the reward distribution are incorporated into the decision process.

[0016] Preferably, A multi-objective optimization module including time constraints, space constraints and resource constraints is constructed. The multi-objective optimization module optimizes the task allocation scheme based on an improved particle swarm algorithm. The steps of obtaining the optimal task allocation scheme include: Define multiple objective functions, including minimizing the total rescue time, maximizing resource utilization, and maximizing the rescue success rate; set time constraints, space constraints, and resource constraints, corresponding to the maximum allowable rescue time, available space resources, and total available resources, respectively; An improved multi-objective particle swarm optimization algorithm is used for multi-objective optimization, and the improvements include: Introducing an adaptive inertia weight update mechanism based on particle fitness and iteration number to dynamically balance global exploration and local development capabilities; Use the non-dominated sorting method based on crowding degree to update the individual optimal solution and the global optimal solution; The ε constraint method is used to handle the constraints, and the degree of constraint violation is included in the particle fitness evaluation; Introducing a neighborhood learning mechanism based on Gaussian perturbation; Implement adaptive mutation operations to dynamically adjust the mutation intensity based on the search process and population status; During the optimization process, non-dominated solutions are stored through an external archive set, and crowding distance is used as the selection criterion; Gradually reduce the variance of Gaussian perturbations to achieve a smooth transition from global exploration to local fine search; Adjust the mutation probability according to the average diversity index of the current population; Iterate the optimization steps until the preset termination condition is reached; Select the optimal task allocation solution from the final non-dominated solution set.

[0017] The second aspect of the present invention, Provides an automated task allocation system for elevator emergency rescue platforms, including: The first unit is used to obtain elevator operation status data and environmental information data, perform data cleaning and standardization on the elevator operation status data and the environmental information data to obtain pre-processed data; based on the pre-processed data, construct an emergency task model, classify the emergency task model, and obtain classified emergency tasks; The second unit is used to input the classified emergency tasks into the deep reinforcement learning model to generate a task allocation plan; the deep reinforcement learning model is composed of a deep Q network structure combined with a priority scanning algorithm, and introduces a priority experience playback mechanism; the state space of the deep reinforcement learning model includes currently available rescue resources, task urgency and geographic location information, and the action space includes rescue personnel selection and path planning decisions; based on the state space and action space, a reward function is designed, and the reward function takes into account task completion time, resource utilization and rescue success rate; The third unit is used to construct a multi-objective optimization module including time constraints, space constraints and resource constraints. The multi-objective optimization module optimizes the task allocation plan based on the improved particle swarm algorithm to obtain the optimal task allocation plan; based on the optimal task allocation plan, a task execution plan is generated; the optimal task allocation plan is sent to the real-time monitoring module, and the real-time monitoring module monitors the task execution process and collects feedback information; the feedback information is input into the deep reinforcement learning model for online learning and model updating to achieve continuous optimization of the task allocation plan.

[0018] The third aspect of the present invention, An electronic device is provided, comprising: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0019] A fourth aspect of the present invention, A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0020] The design of the deep reinforcement learning model adopted in this paper has many beneficial effects. First, the deep Q network structure can effectively process high-dimensional state space and extract complex feature representations. The introduction of the attention mechanism enables the model to focus on the most relevant information and improve the accuracy of decision-making. Secondly, the priority scanning algorithm greatly reduces the search space and improves the decision-making efficiency by processing the action space in layers, which is particularly suitable for dealing with problems in large-scale action spaces. Thirdly, the priority experience replay mechanism can more effectively utilize important experience, accelerate the learning process, and improve sample utilization efficiency. Finally, the application of various optimization techniques such as batch normalization, gradient clipping, and learning rate adjustment enhances the stability and generalization ability of the model. In summary, this scheme can achieve efficient learning and accurate decision-making in complex decision-making environments, and provides a powerful tool for solving complex problems such as elevator emergency dispatch.

[0021] The fusion complex loss function design adopted in this paper has many beneficial effects. First, by introducing the adaptive Huber loss, the robustness of the model to outliers is enhanced, making the learning process more stable. Secondly, the design of the importance sampling weights takes into account multiple factors, which can better reflect the true importance of the samples and improve the learning efficiency. Thirdly, the multi-objective weighting method allows for flexible trade-offs between different optimization objectives to meet the needs of different scenarios. Finally, by introducing uncertainty estimation, the model can dynamically adjust its behavior according to the current learning state, further improving the adaptability and efficiency of learning. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a flow chart of an automated task allocation method of an elevator emergency rescue platform according to an embodiment of the present invention; Figure 2 It is a structural schematic diagram of an automated task allocation system for an elevator emergency rescue platform according to an embodiment of the present invention. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0025] Figure 1 FIG. 1 is a flow chart of an automated task allocation method for an elevator emergency rescue platform according to an embodiment of the present invention. Figure 1 As shown, the method includes: S1. Obtain elevator operation status data and environmental information data, clean and standardize the elevator operation status data and environmental information data to obtain preprocessed data; construct an emergency task model based on the preprocessed data, classify the emergency task model, and obtain classified emergency tasks; S2. Input the classified emergency tasks into the deep reinforcement learning model to generate a task allocation plan; the deep reinforcement learning model is composed of a deep Q network structure combined with a priority scanning algorithm, and introduces a priority experience playback mechanism; the state space of the deep reinforcement learning model includes currently available rescue resources, task urgency and geographic location information, and the action space includes rescue personnel selection and path planning decisions; based on the state space and action space, a reward function is designed, and the reward function takes into account task completion time, resource utilization and rescue success rate; S3. Construct a multi-objective optimization module including time constraints, space constraints and resource constraints. The multi-objective optimization module optimizes the task allocation plan based on the improved particle swarm algorithm to obtain the optimal task allocation plan; based on the optimal task allocation plan, generate a task execution plan; send the optimal task allocation plan to the real-time monitoring module, the real-time monitoring module monitors the task execution process and collects feedback information; input the feedback information into the deep reinforcement learning model for online learning and model updating, so as to achieve continuous optimization of the task allocation plan.

[0026] In an optional embodiment, The steps of constructing an emergency task model based on the preprocessed data and classifying the emergency task model to obtain the classified emergency tasks include: Receiving pre-processed elevator operation status data and environmental information data, including time series data, image data and audio data; Using a long short-term memory network to extract features from time series data, the long short-term memory network includes an input layer, two long short-term memory layers and a fully connected layer, wherein the two long short-term memory layers have different numbers of units, and each long short-term memory layer is set with dropout; Using a pre-trained residual neural network to extract features from image data, the residual neural network removes the original fully connected layer and adds a global average pooling layer and a new fully connected layer; Convert the audio data into a Mel-spectrogram, and use a convolutional neural network to extract features from the Mel-spectrogram, wherein the convolutional neural network includes two convolutional layers, two maximum pooling layers, and one fully connected layer; Apply the self-attention mechanism to the features of time series data, image data, and audio data, calculate the attention scores and obtain weighted features; The weighted features are concatenated and fused through a multi-layer perceptron to obtain fused features; Based on the fusion features, a hierarchical attention network is constructed, wherein the hierarchical attention network includes an urgency classification layer, a fault type classification layer, and a specific problem classification layer; Apply the attention mechanism in each classification layer, calculate the attention weight and obtain the weighted context vector; Use the softmax classifier to perform multi-category classification on the weighted context vector of each classification layer; A multi-task learning framework is used to combine the cross entropy loss of each classification layer to construct a comprehensive loss function; Training the hierarchical attention network using an optimizer and applying an early stopping strategy and a regularization method during the training process; The classification results of the trained hierarchical attention network are post-processed, and the post-processed classification results are output to obtain the classified emergency tasks.

[0027] For example, building an emergency task model based on preprocessed data and classifying it is a complex process involving multiple data types and deep learning techniques. The core of this technical solution is to use multimodal data fusion and hierarchical attention mechanism to achieve efficient and accurate emergency task classification.

[0028] First, the system receives pre-processed elevator operation status data and environmental information data, which include time series data, image data, and audio data. Time series data may include information that changes over time, such as elevator speed, acceleration, and door switch status; image data may come from surveillance cameras inside the elevator and the surrounding environment; audio data may include elevator operation sounds, passenger conversations, or alarm sounds.

[0029] For time series data, a long short-term memory network (LSTM) is used for feature extraction. The network consists of an input layer, two LSTM layers, and a fully connected layer. The two LSTM layers are set with 64 and 32 units respectively to capture time dependencies of different scales. The dropout technique is applied after each LSTM layer, and the dropout rate is set to 0.5 to prevent overfitting. For example, for a 10-second elevator operation data, the sampling frequency is 100Hz and the input dimension is 1000×10 (1000 time steps, 10 features per step). After processing by the LSTM network, a 128-dimensional feature vector is obtained.

[0030] The image data is processed using a pre-trained ResNet-50 model. The original fully connected layer is removed, and a global average pooling layer and a new fully connected layer (output dimension is 256) are added. For example, for an RGB image with a resolution of 224×224×3, after being processed by the modified ResNet-50, a 256-dimensional feature vector is finally obtained.

[0031] The audio data is first converted into a Mel spectrogram. Specifically, a short-time Fourier transform is performed on the audio signal, the frequency is converted to the Mel scale, and the logarithmic amplitude is taken. For example, for a 5-second audio, the sampling rate is 44.1kHz, which is converted into a Mel spectrogram of size 128×217. A convolutional neural network (CNN) is then used for feature extraction. The CNN includes two convolutional layers, two maximum pooling layers, and one fully connected layer. The first convolutional layer uses 32 3×3 convolution kernels, and the second convolutional layer uses 64 3×3 convolution kernels. The pooling size of the maximum pooling layer is 2×2. The output dimension of the fully connected layer is 128.

[0032] Next, a self-attention mechanism is applied to each modality feature. When calculating the attention score, the scaled dot product attention method is used. For example, for time series features, query, key, and value matrices are generated, and their dimensions are all 128×128. The attention weights are calculated through matrix multiplication and softmax operation, and then the weighted features are obtained. This process is also applied to image and audio features.

[0033] The weighted features of each modality are concatenated into a 512-dimensional vector (128+256+128), and then the features are fused through a three-layer multilayer perceptron. The hidden layer sizes of the multilayer perceptron are 256 and 128 respectively, and finally a 128-dimensional fused feature vector is output.

[0034] Based on the fused features, a hierarchical attention network is constructed. The network consists of three classification layers: urgency classification layer, fault type classification layer, and specific problem classification layer. The attention mechanism is applied to each classification layer. Taking urgency classification as an example, the fused features are first processed through a bidirectional GRU network, and then the attention weights are calculated to obtain the weighted context vector.

[0035] A softmax classifier is used at each classification layer for multi-category classification. For example, the urgency may be divided into three categories: "very urgent", "urgent", and "general"; the fault type may include "mechanical fault", "electrical fault", "software fault", etc.; the specific problem may be subdivided into "door fault", "button fault", "brake system fault", etc.

[0036] A multi-task learning framework is used to combine the cross entropy losses of each classification layer into a comprehensive loss function. Different weights can be assigned to different tasks, for example, the weight of the urgency classification may be higher. The network is trained using the Adam optimizer, with the initial learning rate set to 0.001 and a learning rate decay strategy. During the training process, an early stopping strategy is applied, that is, training is stopped when the performance on the validation set has not improved for 5 consecutive epochs. At the same time, the L2 regularization method is used, and the regularization coefficient is set to 0.0001 to prevent overfitting.

[0037] Finally, the classification results of the trained hierarchical attention network are post-processed. This may include rule filtering, confidence threshold screening, etc. For example, if the classification confidence of a specific question is lower than 0.5, it is marked as "unknown question". The classification result after post-processing is the final emergency task classification result.

[0038] This paper makes full use of time series, image and audio data through multimodal data fusion to improve the comprehensiveness and accuracy of classification. Secondly, the hierarchical attention mechanism can automatically focus on important features and improve the interpretability of the model. Thirdly, the multi-task learning framework enables the model to handle multiple related tasks at the same time, improving efficiency and enhancing the generalization ability of the model. Finally, various optimization techniques such as dropout, early stopping and regularization are used to effectively prevent overfitting and improve the robustness of the model in practical applications.

[0039] In an optional embodiment, The deep reinforcement learning model is composed of a deep Q network structure combined with a priority scanning algorithm, and the steps of introducing a priority experience replay mechanism include: Constructing a deep Q network, wherein the deep Q network includes an input layer, multiple fully connected layers, an attention layer, and an output layer; establishing a main network and a target network, wherein the main network is used to select actions, and the target network is used to calculate target Q values, and periodically updating the target network parameters by soft updating; Design a priority scanning algorithm, divide the action space into layers, and design a scoring function for each action layer; The deep Q network is combined with the priority scanning algorithm, including: using the deep Q network to forward propagate the current state to obtain a preliminary Q value estimate; using the priority scanning algorithm to calculate the action priority score layer by layer starting from the top layer, where the scoring function comprehensively considers the preliminary Q value estimate and the action characteristics; selecting the top-k highest-scoring actions at each layer, and the score is the weighted sum of the preliminary Q value estimate and the action priority score; at the bottom layer, selecting the action with the highest comprehensive score from the remaining candidate actions as the final decision; Introduce a priority experience replay mechanism, use the SumTree data structure to store experience tuples and corresponding priorities, calculate the priority based on the TD error, and determine the sampling probability based on the priority; calculate the importance sampling weight; sample training data from the experience pool to train the deep reinforcement learning model; The ε-greedy strategy is used for motion exploration, and the ε value is dynamically adjusted during training; the adaptive Huber loss function is used in combination with the importance sampling weights to calculate the overall loss; the network parameters are updated through the Adam optimizer; the gradient clipping technique is applied during training to limit the L2 norm of the gradient; a batch normalization layer is added after each fully connected layer; the cosine annealing strategy is used to dynamically adjust the learning rate; the priority of the sampling experience is updated according to the new TD error obtained during training; According to the deep Q network structure, priority scanning algorithm and priority experience replay mechanism, the deep reinforcement learning model is iteratively trained until the deep reinforcement learning model converges or reaches the preset number of training times.

[0040] For example, the construction of a deep reinforcement learning model is a complex and sophisticated process that combines a deep Q-network structure, a priority scanning algorithm, and a priority experience replay mechanism. This comprehensive solution aims to improve the learning efficiency and decision-making quality of the model.

[0041] First, a deep Q network is constructed. The network consists of an input layer, multiple fully connected layers, an attention layer, and an output layer. Specifically, the input layer receives a state vector, which may have a dimension of 128; followed by three fully connected layers, each containing 256, 128, and 64 neurons, respectively, and using ReLU as the activation function. The attention layer uses a multi-head self-attention mechanism, with the number of heads set to 4 and the dimension of each head being 16. Finally, the number of neurons in the output layer is equal to the size of the action space, for example, 64 possible actions. The main network and the target network are established at the same time, and they have the same structure. The main network is used to select actions, and the target network is used to calculate the target Q value. The parameters of the target network are regularly updated by soft updating, and the update rate τ is set to 0.01, that is, each time the update occurs, the target network parameters move 1% toward the main network parameters.

[0042] Next, design a priority scanning algorithm. This algorithm divides the action space into multiple levels, such as three levels: high-level strategy, mid-level operation, and low-level control. Design a scoring function for each level, which considers factors such as the importance, urgency, and resource consumption of the action. For example, the scoring function of the high-level strategy layer may focus more on long-term benefits, while the scoring function of the low-level control layer may focus more on immediate feedback.

[0043] Combining the deep Q-network with the priority scanning algorithm is a key step. First, the current state is forward propagated using the deep Q-network to obtain a preliminary Q-value estimate. Then, the priority scanning algorithm is used to calculate the action priority scores layer by layer starting from the top layer. At the high-level policy layer, the top-3 highest-scoring actions may be selected; at the mid-level operation layer, the top-5 highest-scoring actions are selected for each high-level policy; at the low-level control layer, the action with the highest comprehensive score is selected from the remaining candidate actions as the final decision. The comprehensive score is the weighted sum of the preliminary Q-value estimate and the action priority score, and the weights can be set to 0.7 and 0.3.

[0044] Introducing a priority experience replay mechanism is an important means to improve learning efficiency. Use the SumTree data structure to store experience tuples and corresponding priorities. The number of leaf nodes of SumTree can be set to 100,000. Calculate the priority based on the TD error. The absolute value of the TD error plus a small constant (such as 0.01) is used as the priority. Determine the sampling probability based on the priority. The sampling probability is proportional to the 0.6 power of the priority. When calculating the importance sampling weight, use the inverse of the sampling probability and normalize the weight to the interval [0, 1]. Sample a batch of training data (such as 256 samples) from the experience pool for deep reinforcement learning model training.

[0045] During the training process, the ε-greedy strategy is used for motion exploration. The initial ε value is set to 1.0, which gradually decays to 0.01 with training, and the decay rate is 0.995. The adaptive Huber loss function is combined with the importance sampling weight to calculate the overall loss. The initial value of the δ parameter of the Huber loss is set to 1.0 and dynamically adjusted during the training process. The network parameters are updated by the Adam optimizer, and the initial learning rate is set to 0.001. The gradient clipping technique is applied to limit the L2 norm of the gradient to a maximum of 10. A batch normalization layer is added after each fully connected layer, and the momentum parameter is set to 0.99. The cosine annealing strategy is used to dynamically adjust the learning rate, with a period of 10,000 steps and a minimum learning rate of 1% of the initial learning rate. The priority of the sampled experience is updated according to the new TD error obtained during the training process, and the update frequency is once every 10 steps.

[0046] Finally, the deep reinforcement learning model is iteratively trained based on the deep Q network structure, priority scanning algorithm, and priority experience replay mechanism. Each training round consists of 1000 time steps, and a total of 100,000 rounds are trained or until the model converges. The convergence criterion can be set as the average return change for 100 consecutive rounds is less than 1%. During the training process, the model performance is evaluated every 1000 rounds, and the exploration strategy is turned off during the evaluation, and decisions are made purely based on the learned strategy.

[0047] The design of the deep reinforcement learning model adopted in this paper has many beneficial effects. First, the deep Q network structure can effectively process high-dimensional state space and extract complex feature representations. The introduction of the attention mechanism enables the model to focus on the most relevant information and improve the accuracy of decision-making. Secondly, the priority scanning algorithm greatly reduces the search space and improves the decision-making efficiency by processing the action space in layers, which is particularly suitable for dealing with problems in large-scale action spaces. Thirdly, the priority experience replay mechanism can more effectively utilize important experience, accelerate the learning process, and improve sample utilization efficiency. Finally, the application of various optimization techniques such as batch normalization, gradient clipping, and learning rate adjustment enhances the stability and generalization ability of the model. In summary, this scheme can achieve efficient learning and accurate decision-making in complex decision-making environments, and provides a powerful tool for solving complex problems such as elevator emergency dispatch.

[0048] In an optional embodiment, The ε-greedy strategy is used for action exploration. The steps of dynamically adjusting the ε value during training include: Initialize the basic ε-greedy strategy, define the state space, action space and state-action value function; An exponential decay mechanism is introduced to dynamically adjust the ε value according to the number of training steps, so that the ε value gradually decreases from the initial high exploration rate to a stable low exploration rate; Establish a performance-driven adaptive adjustment mechanism to increase exploration when learning performance improves and reduce exploration when learning performance decreases; Combining the Thompson sampling idea, a Bayesian neural network is used to estimate the Q value distribution, and the ε value is dynamically adjusted according to the uncertainty of the Q value; A hierarchical ε-greedy strategy is constructed to adopt different exploration rates at different decision levels according to the hierarchical nature of the elevator rescue task. Dynamically calculate and update the ε value in each decision cycle; select exploration or exploitation behavior in the current state based on the updated ε value; execute the selected behavior, observe environmental feedback, and update the state-action value function; based on the new state-action value function and environmental feedback, adjust the mean and standard deviation of the Q value estimate.

[0049] For example, the ε-greedy strategy is a commonly used exploration-exploitation balance method in reinforcement learning. Its core idea is to dynamically adjust the exploration rate during the learning process to achieve a balance between acquiring new knowledge and exploiting existing knowledge. In a complex environment such as an elevator rescue mission, it is particularly important to design an efficient ε-greedy strategy.

[0050] First, initialize the basic ε-greedy strategy. Define the state space, including the current position of the elevator, the operating status, the fault type and other features, and the number of possible states is 10,000. The action space includes rescue operations at different levels, such as "emergency stop", "remote restart", "dispatch maintenance personnel", etc., with a total of 100 possible actions. The state-action value function is initialized as a zero matrix with a size of 10,000x100.

[0051] An exponential decay mechanism is introduced to dynamically adjust the ε value. The initial ε value is set to 1.0, indicating a 100% exploration rate. As the number of training steps increases, the ε value gradually decreases.

[0052] Establish a performance-driven adaptive adjustment mechanism. Define a sliding window (e.g., 1000 steps) to evaluate learning performance. Calculate the average return within the window. If the average return of the current window is 10% higher than that of the previous window, increase the ε value by 0.1 (upper limit is 1.0); if it is 10% lower, decrease the ε value by 0.1 (lower limit is 0.01). This can increase exploration when the learning effect is poor and make more use of existing knowledge when the learning effect is good.

[0053] Combined with the Thompson sampling idea, a Bayesian neural network is used to estimate the Q value distribution. The structure of a Bayesian neural network is similar to that of a deep Q network, but each weight is a probability distribution rather than a fixed value. For example, it can be assumed that each weight follows a normal distribution with an initial mean of 0 and a standard deviation of 0.1. At each decision, a set of specific weight values ​​are sampled from these distributions, and then this set of weights is used for forward propagation to obtain an estimate of the Q value. The uncertainty of the Q value can be expressed by its confidence interval. When the confidence interval is large, increase the ε value to encourage exploration; when the confidence interval is small, reduce the ε value to make more use of existing knowledge.

[0054] Construct a hierarchical ε-greedy strategy. In view of the hierarchical nature of the elevator rescue task, different exploration rates can be used at different decision levels. For example, decisions can be divided into three levels: strategic level (such as whether manual intervention is needed), tactical level (such as which type of rescue plan to choose), and operational level (such as specific operational steps). Use a lower ε value (such as 0.1) at the strategic level because the cost of wrong decisions is high; use a medium ε value (such as 0.3) at the tactical level; and use a higher ε value (such as 0.5) at the operational level to encourage trying different specific operations.

[0055] In each decision cycle, the ε value is dynamically calculated and updated. The final ε value is obtained by comprehensively considering exponential decay, performance-driven adjustments, Thompson sampling results, and decision levels. For example, the impact of these factors can be weighted averaged with weights of 0.4, 0.3, 0.2, and 0.1. According to the updated ε value, choose to explore or exploit in the current state. Specifically, generate a random number between [0,1]. If it is less than ε, choose a random action (explore), otherwise choose the action with the current best estimate (exploit).

[0056] Perform the selected behavior and observe the environmental feedback. Environmental feedback may include indicators such as the success rate of the rescue operation, rescue time, resource consumption, etc. Update the state-action value function based on these feedbacks. For example, if the Q-learning algorithm is used, the temporal difference (TD) error can be used to update the Q value.

[0057] Based on the new state-action value function and the environment feedback, adjust the mean and standard deviation of the Q-value estimate. For Bayesian neural networks, variational inference methods can be used to update the posterior distribution of network weights. For example, the KL divergence between the predicted distribution and the true return distribution can be minimized to update the mean and standard deviation through backpropagation.

[0058] The ε-greedy strategy adopted in this paper has many beneficial effects. First, the exponential decay mechanism ensures sufficient exploration in the early stage of learning, and more use of the learned knowledge in the later stage helps to quickly converge to a good strategy. Second, the performance-driven adaptive adjustment enables the exploration rate to change dynamically according to the learning effect, improving the efficiency and stability of learning. Third, the introduction of Thompson sampling takes into account the uncertainty of decision-making, increases exploration when more information is needed, and improves the pertinence of learning. Finally, the hierarchical ε-greedy strategy takes into account the hierarchical nature of the task and adopts different exploration strategies at different decision-making levels, which not only ensures the stability of key decisions, but also maintains sufficient flexibility in detailed operations. Overall, this scheme can achieve efficient learning and decision-making in complex elevator rescue tasks, balance exploration and utilization, and improve the performance and adaptability of the reinforcement learning model.

[0059] In an optional embodiment, The steps to calculate the overall loss using the adaptive Huber loss function combined with the importance sampling weights include: The formula for the overall loss function is: ; in: ; ; ; Among them: Total_Loss represents the overall loss function; β j represents the weight coefficient of the jth target; w i represents the importance sampling weight of the i-th sample; L(δ i ,α(t)) represents the adaptive Huber loss function;δ i The parameter represents the prediction error of the i-th sample; α(t) represents the adaptive threshold; α base represents the basic threshold; γ represents the uncertainty adjustment coefficient; std(TD error (t)) represents the standard deviation of the TD error of the current batch; w max represents the upper limit of the importance sampling weight; p i represents the sampling probability of the i-th sample in the priority experience playback; λ represents the weight coefficient of the task urgency; E i represents the urgency of the task related to the i-th sample; μ represents the weight coefficient of the sample novelty; N i represents the novelty score of the i-th sample; max(w) represents the maximum value of the importance sampling weights of all samples; i represents the index of the sample, and j represents the index of the target or task.

[0060] For example, the elevator emergency rescue task allocation problem is a complex multi-objective optimization problem that requires considering multiple factors and balancing different objectives. In order to effectively solve this problem, a fusion complex loss function is proposed, which integrates multiple key elements, including adaptive Huber loss, importance sampling, multi-objective weighting and uncertainty estimation.

[0061] First, define the overall loss function. This function consists of multiple sub-goals, each with its corresponding weight coefficient. For example, three sub-goals can be set: rescue efficiency, resource utilization, and safety, with weight coefficients of 0.4, 0.3, and 0.3, respectively. The loss of each sub-goal is calculated by weighted average, where the weight is the importance of the sample.

[0062] Next, the adaptive Huber loss function is introduced. The core idea of ​​this function is to use different treatment methods for small errors and large errors to reduce the impact of outliers. Specifically, when the absolute value of the prediction error is less than a certain threshold, the square error is used; when the absolute value of the prediction error is greater than the threshold, the linear error is used. This threshold is dynamically adjusted based on the standard deviation of the TD error of the current batch. For example, the basic threshold can be set to 1.0 and the uncertainty adjustment coefficient to 0.5. If the standard deviation of the TD error of the current batch is 0.2, the threshold actually used is 1.1.

[0063] The calculation of the importance sampling weight is the next key step. This weight takes into account multiple factors: the sampling probability of the sample in the priority experience replay, the urgency of the task, and the novelty of the sample. For example, for a specific sample, if its sampling probability is 0.01, the urgency score of the relevant task is 0.8 (full score is 1), and the novelty score is 0.6 (full score is 1), and the weight coefficient of the task urgency is set to 0.3 and the weight coefficient of the novelty is set to 0.2, then the preliminary importance weight of the sample may be 0.01 * 1 + 0.3 * 0.8 + 0.2 * 0.6 = 0.37. Finally, the weights of all samples are normalized and an upper limit (such as 5.0) is set to prevent some samples from having too large weights.

[0064] In practical applications, the calculation process of this fusion loss function can be divided into the following steps: First, the prediction error is calculated for each example. This can be done by comparing the Q-value predicted by the model and the actual observed reward. For example, if the model predicts a Q-value of 10 for an action and the actual observed reward is 12, the prediction error is 2.

[0065] Then, the standard deviation of the TD error of the current batch is calculated. Assuming that a batch contains 32 samples, the standard deviation of these 32 prediction errors is calculated to be 0.2. Based on this standard deviation and the pre-set parameters, the threshold of the adaptive Huber loss function is calculated, such as 1.1.

[0066] Next, the adaptive Huber loss function is applied to each sample. For samples with a prediction error of 2, since 2 is greater than the threshold 1.1, the linear error calculation method will be used.

[0067] Calculate the importance sampling weight of each sample. This needs to take into account the sampling probability, task urgency, and novelty of the sample. For example, for the sample mentioned above, its importance sampling weight is 0.37.

[0068] Multiply the calculated loss value by the corresponding importance sampling weight, and then take the weighted average of all samples to get the loss value of a sub-goal.

[0069] Repeat the above process to calculate the loss values ​​of all sub-goals. Finally, multiply the loss values ​​of these sub-goals by their respective weight coefficients (such as rescue efficiency 0.4, resource utilization 0.3, safety 0.3), and sum them up to get the final overall loss value.

[0070] The fusion complex loss function design adopted in this paper has many beneficial effects. First, by introducing the adaptive Huber loss, the robustness of the model to outliers is enhanced, making the learning process more stable. Secondly, the design of the importance sampling weights takes into account multiple factors, which can better reflect the true importance of the samples and improve the learning efficiency. Thirdly, the multi-objective weighting method allows for flexible trade-offs between different optimization objectives to meet the needs of different scenarios. Finally, by introducing uncertainty estimation, the model can dynamically adjust its behavior according to the current learning state, further improving the adaptability and efficiency of learning.

[0071] In an optional embodiment, Based on the state space and the action space, a reward function is designed, wherein the reward function considers task completion time, resource utilization rate and rescue success rate, and the steps include: Construct a reward function that combines the task completion time reward, resource utilization reward, and rescue success rate reward in a weighted manner; An exponential decay function based on time pressure is used to calculate the task completion time reward, and the reward value is dynamically adjusted according to the estimated task completion time; The resource utilization rate reward is calculated by combining the utilization rate of rescuers and equipment, and the overall resource utilization efficiency is calculated by arithmetic average; The sigmoid function based on risk assessment is used to calculate the rescue success rate reward, and the reward sensitivity is dynamically adjusted according to the difference between the estimated rescue success probability and the preset threshold; An adaptive weight mechanism is introduced to automatically adjust the weights of each component in the reward function according to the real-time change rate of each indicator during the rescue process; Use Pareto optimization method to solve the reward function and automatically adjust the trade-offs between different objectives; The reward distribution for each action is estimated through the Monte Carlo sampling method, and the mean and variance of the reward distribution are incorporated into the decision process.

[0072] For example, the reward function design for elevator emergency rescue tasks is a complex multi-objective optimization problem that needs to consider multiple factors such as task completion time, resource utilization, and rescue success rate. In order to effectively solve this problem, a comprehensive reward function design method is proposed, which integrates multiple advanced technologies to achieve the optimal balance of various indicators.

[0073] First, build a basic reward function framework. This function consists of three main components: task completion time reward, resource utilization reward, and rescue success rate reward. These three parts are combined in a weighted manner, and the initial weights can be set equal, for example, one-third each. As the rescue mission progresses, these weights will be dynamically adjusted according to the actual situation.

[0074] For the calculation of the task completion time reward, an exponential decay function based on time pressure is used. Specifically, a benchmark completion time can be set, such as 30 minutes. If the actual completion time is less than this benchmark time, the reward value will increase exponentially as time decreases; conversely, the reward value will decrease exponentially as time increases. For example, if the task is completed within 20 minutes, you may get 1.5 times the basic reward; if it takes 40 minutes, you may only get 0.7 times the basic reward. This design encourages the completion of tasks quickly, but also avoids excessive pursuit of speed and ignoring other factors.

[0075] The calculation of resource utilization bonus takes into account both rescuer utilization and equipment utilization. Rescuer utilization can be measured by the ratio of actual working time to total available time, while equipment utilization can be measured by the ratio of actual use time to total available time. For example, if a 4-person rescue team has an average of 1.5 hours of actual working time per person during a 2-hour rescue, then the personnel utilization rate is 75%. Similarly, if a key equipment is used for 1.8 hours within 2 hours, its utilization rate is 90%. The final resource utilization bonus can be taken as the arithmetic mean of these two indicators, which is 82.5% in this example.

[0076] The calculation of the rescue success rate reward uses a sigmoid function based on risk assessment. First, the success probability of the rescue operation needs to be estimated, which can be done based on the current situation, historical data, and expert experience. Then, this estimated probability is compared with the preset success threshold (for example, 80%). When the estimated success rate approaches or exceeds this threshold, the reward value will increase rapidly; conversely, the reward value will decrease rapidly. For example, if the estimated success rate is 85%, you may get a full score reward of 0.9; if the estimated success rate is 70%, you may only get a reward of 0.3. This design encourages actions with high success rates, but also allows certain risks to be taken when necessary.

[0077] In order to make the reward function adaptable to different rescue scenarios and dynamically changing situations, an adaptive weight mechanism is introduced. This mechanism automatically adjusts the weights of each component in the reward function according to the real-time change rate of each indicator during the rescue process. For example, if the improvement rate of task completion time is significantly faster than other indicators in a certain period of time, the system will appropriately increase the weight of the task completion time reward to encourage continued high efficiency. Specifically, a sliding window (such as the last 10 minutes) can be used to calculate the change rate of each indicator, and then the weights can be adjusted according to the relative size of these change rates.

[0078] In the process of solving the reward function, the Pareto optimization method is used. This method can find the best balance between multiple potentially conflicting objectives, avoiding excessive sacrifice of one aspect to improve another aspect. In specific implementation, a multi-objective evolutionary algorithm can be used, such as NSGA-II (Non-dominated Sorting Genetic Algorithm II). This algorithm generates a series of non-dominated solutions, each of which represents a different target trade-off scheme. Decision makers can choose the one that best suits the current situation from these solutions.

[0079] Finally, in order to more fully evaluate the potential impact of each action, the Monte Carlo sampling method is used to estimate the reward distribution. Specifically, for each possible action, the simulation is performed multiple times (such as 1000 times), and each simulation is based on the current state and some random factors. Through these simulations, the distribution of the reward value of the action can be obtained, including the mean and variance. When making decisions, not only the expected reward (ie, the mean) but also the risk (ie, the variance) is considered. For example, among two actions with similar expected rewards, the one with a smaller variance may be chosen to obtain a more stable result.

[0080] The comprehensive reward function design method adopted in this paper has many beneficial effects. First, it comprehensively considers multiple key indicators of elevator emergency rescue tasks and can more accurately reflect the overall performance of the task. Secondly, by introducing the adaptive weight mechanism and Pareto optimization method, the reward function can flexibly adapt to different rescue scenarios and dynamically changing situations, improving the adaptability and robustness of decision-making. Thirdly, the reward calculation method for rescue success rate based on risk assessment helps to encourage necessary risk-taking while ensuring safety. Finally, the application of Monte Carlo sampling method enables the decision-making process to consider uncertainty more comprehensively, so as to make more cautious and reasonable choices. In general, this method provides a comprehensive, flexible and efficient decision support framework for elevator emergency rescue tasks, which is expected to significantly improve the overall effectiveness and reliability of rescue operations.

[0081] In an optional embodiment, A multi-objective optimization module including time constraints, space constraints and resource constraints is constructed. The multi-objective optimization module optimizes the task allocation scheme based on an improved particle swarm algorithm. The steps of obtaining the optimal task allocation scheme include: Define multiple objective functions, including minimizing the total rescue time, maximizing resource utilization, and maximizing the rescue success rate; set time constraints, space constraints, and resource constraints, corresponding to the maximum allowable rescue time, available space resources, and total available resources, respectively; An improved multi-objective particle swarm optimization algorithm is used for multi-objective optimization, and the improvements include: Introducing an adaptive inertia weight update mechanism based on particle fitness and iteration number to dynamically balance global exploration and local development capabilities; Use the non-dominated sorting method based on crowding degree to update the individual optimal solution and the global optimal solution; The ε constraint method is used to handle the constraints, and the degree of constraint violation is included in the particle fitness evaluation; Introducing a neighborhood learning mechanism based on Gaussian perturbation; Implement adaptive mutation operations to dynamically adjust the mutation intensity based on the search process and population status; During the optimization process, non-dominated solutions are stored through an external archive set, and crowding distance is used as the selection criterion; Gradually reduce the variance of Gaussian perturbations to achieve a smooth transition from global exploration to local fine search; Adjust the mutation probability according to the average diversity index of the current population; Iterate the optimization steps until the preset termination condition is reached; Select the optimal task allocation solution from the final non-dominated solution set.

[0082] For example, the elevator emergency rescue task allocation problem is a typical multi-objective optimization problem involving multiple conflicting objectives and complex constraints. In order to effectively solve this problem, an optimization method based on an improved multi-objective particle swarm algorithm is proposed, which integrates a variety of advanced technologies to obtain the optimal task allocation solution.

[0083] First, define multiple objective functions. These objective functions include minimizing the total rescue time, maximizing resource utilization, and maximizing the rescue success rate. For example, the total rescue time can be defined as the total time from receiving the alarm to completing the rescue, and the goal is to minimize this time. Resource utilization can be defined as the ratio of actually used resources to the total available resources, and the goal is to make this ratio as close to 1 as possible. The rescue success rate can be estimated based on historical data and current conditions, and the goal is to make this probability as high as possible.

[0084] At the same time, set corresponding constraints. The time constraint can be set as the maximum allowable rescue time, such as 2 hours. The space constraint can be expressed as the available rescue channel width or the size limit of the elevator shaft, such as the minimum channel width must not be less than 0.8 meters. Resource constraints can include the number of available rescue personnel and equipment, such as a maximum of 10 rescue personnel and 5 professional equipment can be mobilized.

[0085] On this basis, an improved multi-objective particle swarm optimization algorithm is used for multi-objective optimization. The core idea of ​​this algorithm is to simulate swarm intelligence and find the optimal solution through the movement and interaction of a large number of particles in the solution space. Each particle represents a possible task allocation scheme, and its position and speed are continuously updated with the iterative process.

[0086] In order to improve the performance of the algorithm, a number of improvements were introduced. The first is an adaptive inertia weight update mechanism. The inertia weight determines the degree to which the particle maintains its current state of motion, and has an important impact on the algorithm's global exploration and local development capabilities. By introducing a dynamic adjustment mechanism based on particle fitness and the number of iterations, these two capabilities can be automatically balanced during the search process. For example, in the initial stage, a larger inertia weight (such as 0.9) can be set to encourage global exploration; as the iterations proceed, the inertia weight can be gradually reduced (such as to 0.4) to enhance local development capabilities.

[0087] When updating the individual optimal solution and the global optimal solution, a non-dominated sorting method based on crowding is used. This method not only considers the non-dominated relationship of the solution, but also the distribution of the solution, which helps to maintain the diversity of the solution set. Specifically, the solution space can be divided into several equal parts, and the crowding distance of each solution on each target is calculated, and the solution with smaller crowding degree is selected first.

[0088] To handle constraints, the ε constraint method is used. This method incorporates the degree of constraint violation into the fitness evaluation of the particle. For example, if a solution violates the time constraint and exceeds the maximum allowed time by 10 minutes, a penalty term can be added to its fitness, and the penalty value is proportional to the excess time.

[0089] The introduction of a neighborhood learning mechanism based on Gaussian perturbation can enhance the local search capability of the algorithm. In specific implementation, after each update of the particle position, the particle can be Gaussian perturbed with a certain probability (such as 0.1). The standard deviation of the perturbation can decrease with the increase of the number of iterations, achieving a smooth transition from global exploration to local fine search.

[0090] Adaptive mutation operation is another important improvement. Dynamically adjusting the mutation intensity according to the search process and population status can strike a balance between maintaining population diversity and accelerating convergence. For example, a base mutation probability (such as 0.01) can be set and then adjusted according to the average diversity index of the current population. If the diversity index is below a certain threshold, the mutation probability can be appropriately increased.

[0091] During the optimization process, non-dominated solutions are stored in an external archive set, and crowding distance is used as the selection criterion. This keeps all non-dominated solutions found during the search process, providing more options for the final solution selection. When the archive set capacity reaches the upper limit, some solutions can be deleted based on the crowding distance to retain a more evenly distributed solution set.

[0092] The algorithm's iterative process continues until a preset termination condition is reached, which can include a maximum number of iterations (such as 1,000), a maximum computational time (such as 1 hour), or no significant improvement after multiple consecutive iterations.

[0093] Finally, the optimal task allocation scheme is selected from the final set of non-dominated solutions. This step usually requires decision makers to weigh and choose according to the specific situation. Some auxiliary tools, such as visualization tools, can be provided to help decision makers intuitively compare the performance of different schemes on various objectives.

[0094] The optimization method based on the improved multi-objective particle swarm algorithm adopted in this paper has many beneficial effects. First, it can consider multiple conflicting objectives at the same time and provide decision makers with a series of balanced non-dominated solutions instead of a single "optimal" solution, which is more in line with the needs of actual decision-making. Secondly, by introducing a number of improvements, such as adaptive inertia weight, non-dominated sorting, and ε constraint method, the search efficiency of the algorithm and the quality of the solution are significantly improved. Thirdly, the method can effectively handle complex constraints and ensure that the generated solutions are feasible. Finally, the adaptive nature of the algorithm enables it to flexibly respond to different problem instances without frequent parameter adjustments. In general, this method provides a powerful and flexible optimization framework for the elevator emergency rescue task allocation problem, which is expected to significantly improve the efficiency and success rate of rescue operations.

[0095] Figure 2 FIG. 1 is a schematic diagram of the structure of the automated task allocation system of the elevator emergency rescue platform according to an embodiment of the present invention. Figure 2 As shown, the system comprises: The first unit is used to obtain elevator operation status data and environmental information data, perform data cleaning and standardization on the elevator operation status data and the environmental information data to obtain pre-processed data; based on the pre-processed data, construct an emergency task model, classify the emergency task model, and obtain classified emergency tasks; The second unit is used to input the classified emergency tasks into the deep reinforcement learning model to generate a task allocation plan; the deep reinforcement learning model is composed of a deep Q network structure combined with a priority scanning algorithm, and introduces a priority experience playback mechanism; the state space of the deep reinforcement learning model includes currently available rescue resources, task urgency and geographic location information, and the action space includes rescue personnel selection and path planning decisions; based on the state space and action space, a reward function is designed, and the reward function takes into account task completion time, resource utilization and rescue success rate; The third unit is used to construct a multi-objective optimization module including time constraints, space constraints and resource constraints. The multi-objective optimization module optimizes the task allocation plan based on the improved particle swarm algorithm to obtain the optimal task allocation plan; based on the optimal task allocation plan, a task execution plan is generated; the optimal task allocation plan is sent to the real-time monitoring module, and the real-time monitoring module monitors the task execution process and collects feedback information; the feedback information is input into the deep reinforcement learning model for online learning and model updating to achieve continuous optimization of the task allocation plan.

[0096] According to a third aspect of the embodiments of the present invention, An electronic device is provided, comprising: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0097] A fourth aspect of the embodiments of the present invention is: A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0098] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.

[0099] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. The automated task allocation method for the elevator emergency rescue platform is characterized in that: include: Acquire elevator operation status data and environmental information data, and perform data cleaning and standardization on the elevator operation status data and environmental information data to obtain pre-processed data; Based on the preprocessed data, an emergency task model is constructed, and the emergency task model is classified to obtain classified emergency tasks; The classified emergency tasks are input into the deep reinforcement learning model to generate a task allocation plan; the deep reinforcement learning model is composed of a deep Q network structure combined with a priority scanning algorithm, and a priority experience replay mechanism is introduced; the state space of the deep reinforcement learning model includes currently available rescue resources, task urgency and geographic location information, and the action space includes rescue personnel selection and path planning decisions; based on the state space and action space, a reward function is designed, and the reward function takes into account task completion time, resource utilization and rescue success rate; Constructing a multi-objective optimization module including time constraints, space constraints and resource constraints, the multi-objective optimization module optimizes the task allocation scheme based on an improved particle swarm algorithm to obtain an optimal task allocation scheme; based on the optimal task allocation scheme, generating a task execution scheme; The optimal task allocation scheme is sent to a real-time monitoring module, which monitors the task execution process and collects feedback information; The feedback information is input into the deep reinforcement learning model for online learning and model updating to achieve continuous optimization of the task allocation plan.

2. The method according to claim 1, characterized in that The steps of constructing an emergency task model based on the preprocessed data and classifying the emergency task model to obtain the classified emergency tasks include: Receiving pre-processed elevator operation status data and environmental information data, including time series data, image data and audio data; Using a long short-term memory network to extract features from time series data, the long short-term memory network includes an input layer, two long short-term memory layers and a fully connected layer, wherein the two long short-term memory layers have different numbers of units, and each long short-term memory layer is set with dropout; Using a pre-trained residual neural network to extract features from image data, the residual neural network removes the original fully connected layer and adds a global average pooling layer and a new fully connected layer; Convert the audio data into a Mel-spectrogram, and use a convolutional neural network to extract features from the Mel-spectrogram, wherein the convolutional neural network includes two convolutional layers, two maximum pooling layers, and one fully connected layer; Apply the self-attention mechanism to the features of time series data, image data, and audio data, calculate the attention scores and obtain weighted features; The weighted features are concatenated and fused through a multi-layer perceptron to obtain fused features; Based on the fusion features, a hierarchical attention network is constructed, wherein the hierarchical attention network includes an urgency classification layer, a fault type classification layer, and a specific problem classification layer; Apply the attention mechanism in each classification layer, calculate the attention weight and obtain the weighted context vector; Use the softmax classifier to perform multi-category classification on the weighted context vector of each classification layer; A multi-task learning framework is used to combine the cross entropy loss of each classification layer to construct a comprehensive loss function; Training the hierarchical attention network using an optimizer and applying an early stopping strategy and a regularization method during the training process; The classification results of the trained hierarchical attention network are post-processed, and the post-processed classification results are output to obtain the classified emergency tasks.

3. The method according to claim 1, characterized in that: The deep reinforcement learning model is composed of a deep Q network structure combined with a priority scanning algorithm, and the steps of introducing a priority experience replay mechanism include: Constructing a deep Q network, wherein the deep Q network includes an input layer, multiple fully connected layers, an attention layer, and an output layer; establishing a main network and a target network, wherein the main network is used to select actions, and the target network is used to calculate target Q values, and periodically updating the target network parameters by soft updating; Design a priority scanning algorithm, divide the action space into layers, and design a scoring function for each action layer; The deep Q network is combined with the priority scanning algorithm, including: using the deep Q network to forward propagate the current state to obtain a preliminary Q value estimate; using the priority scanning algorithm to calculate the action priority score layer by layer starting from the top layer, where the scoring function comprehensively considers the preliminary Q value estimate and the action characteristics; selecting the top-k highest-scoring actions at each layer, and the score is the weighted sum of the preliminary Q value estimate and the action priority score; at the bottom layer, selecting the action with the highest comprehensive score from the remaining candidate actions as the final decision; Introduce a priority experience replay mechanism, use the SumTree data structure to store experience tuples and corresponding priorities, calculate the priority based on the TD error, and determine the sampling probability based on the priority; calculate the importance sampling weight; sample training data from the experience pool to train the deep reinforcement learning model; The ε-greedy strategy is used for motion exploration, and the ε value is dynamically adjusted during training; the adaptive Huber loss function is used in combination with the importance sampling weights to calculate the overall loss; the network parameters are updated through the Adam optimizer; the gradient clipping technique is applied during training to limit the L2 norm of the gradient; a batch normalization layer is added after each fully connected layer; the cosine annealing strategy is used to dynamically adjust the learning rate; the priority of the sampling experience is updated according to the new TD error obtained during training; According to the deep Q network structure, priority scanning algorithm and priority experience replay mechanism, the deep reinforcement learning model is iteratively trained until the deep reinforcement learning model converges or reaches the preset number of training times.

4. The method according to claim 3, characterized in that The ε-greedy strategy is used for action exploration. The steps of dynamically adjusting the ε value during training include: Initialize the basic ε-greedy strategy, define the state space, action space and state-action value function; An exponential decay mechanism is introduced to dynamically adjust the ε value according to the number of training steps, so that the ε value gradually decreases from the initial high exploration rate to a stable low exploration rate; Establish a performance-driven adaptive adjustment mechanism to increase exploration when learning performance improves and reduce exploration when learning performance decreases; Combining the Thompson sampling idea, a Bayesian neural network is used to estimate the Q value distribution, and the ε value is dynamically adjusted according to the uncertainty of the Q value; Construct a hierarchical ε-greedy strategy, which adopts different exploration rates at different decision levels according to the hierarchical nature of the elevator rescue task; Dynamically calculate and update the ε value in each decision cycle; select exploration or exploitation behavior in the current state based on the updated ε value; execute the selected behavior, observe environmental feedback, and update the state-action value function; based on the new state-action value function and environmental feedback, adjust the mean and standard deviation of the Q value estimate.

5. The method according to claim 3, characterized in that: The steps to calculate the overall loss using the adaptive Huber loss function combined with the importance sampling weights include: The formula for the overall loss function is: ； in: ； ； ； Among them: Total_Loss represents the overall loss function; β j represents the weight coefficient of the jth target; w i represents the importance sampling weight of the i-th sample; L(δ i ,α(t)) represents the adaptive Huber loss function;δ i The parameter represents the prediction error of the i-th sample; α(t) represents the adaptive threshold; α base represents the basic threshold; γ represents the uncertainty adjustment coefficient; std(TD error (t)) represents the standard deviation of the TD error of the current batch; w max represents the upper limit of the importance sampling weight; p i represents the sampling probability of the i-th sample in the priority experience playback; λ represents the weight coefficient of the task urgency; E i represents the urgency of the task related to the i-th sample; μ represents the weight coefficient of the sample novelty; N i represents the novelty score of the i-th sample; max(w) represents the maximum value of the importance sampling weights of all samples; i represents the index of the sample, and j represents the index of the target or task.

6. The method according to claim 1, characterized in that Based on the state space and the action space, a reward function is designed, wherein the reward function considers task completion time, resource utilization rate and rescue success rate, and the steps include: Construct a reward function that combines the task completion time reward, resource utilization reward, and rescue success rate reward in a weighted manner; An exponential decay function based on time pressure is used to calculate the task completion time reward, and the reward value is dynamically adjusted according to the estimated task completion time; The resource utilization rate reward is calculated by combining the utilization rate of rescuers and equipment, and the overall resource utilization efficiency is calculated by arithmetic average; The sigmoid function based on risk assessment is used to calculate the rescue success rate reward, and the reward sensitivity is dynamically adjusted according to the difference between the estimated rescue success probability and the preset threshold; An adaptive weight mechanism is introduced to automatically adjust the weights of each component in the reward function according to the real-time change rate of each indicator during the rescue process; Use Pareto optimization method to solve the reward function and automatically adjust the trade-offs between different objectives; The reward distribution for each action is estimated through the Monte Carlo sampling method, and the mean and variance of the reward distribution are incorporated into the decision process.

7. The method according to claim 1, characterized in that A multi-objective optimization module including time constraints, space constraints and resource constraints is constructed. The multi-objective optimization module optimizes the task allocation scheme based on an improved particle swarm algorithm. The steps of obtaining the optimal task allocation scheme include: Define multiple objective functions, including minimizing the total rescue time, maximizing resource utilization, and maximizing the rescue success rate; set time constraints, space constraints, and resource constraints, corresponding to the maximum allowable rescue time, available space resources, and total available resources, respectively; An improved multi-objective particle swarm optimization algorithm is used for multi-objective optimization, and the improvements include: Introducing an adaptive inertia weight update mechanism based on particle fitness and iteration number to dynamically balance global exploration and local development capabilities; Use the non-dominated sorting method based on crowding degree to update the individual optimal solution and the global optimal solution; The ε constraint method is used to handle the constraints, and the degree of constraint violation is included in the particle fitness evaluation; Introducing a neighborhood learning mechanism based on Gaussian perturbation; Implement adaptive mutation operations to dynamically adjust the mutation intensity based on the search process and population status; During the optimization process, non-dominated solutions are stored through an external archive set, and crowding distance is used as the selection criterion; Gradually reduce the variance of Gaussian perturbations to achieve a smooth transition from global exploration to local fine search; Adjust the mutation probability according to the average diversity index of the current population; Iterate the optimization steps until the preset termination condition is reached; Select the optimal task allocation solution from the final non-dominated solution set.

8. An automated task allocation system for an elevator emergency rescue platform, used to implement the method according to any one of claims 1 to 7, characterized in that: include: The first unit is used to obtain elevator operation status data and environmental information data, and perform data cleaning and standardization on the elevator operation status data and environmental information data to obtain pre-processed data; Based on the preprocessed data, an emergency task model is constructed, and the emergency task model is classified to obtain classified emergency tasks; The second unit is used to input the classified emergency tasks into the deep reinforcement learning model to generate a task allocation plan; the deep reinforcement learning model is composed of a deep Q network structure combined with a priority scanning algorithm, and introduces a priority experience playback mechanism; the state space of the deep reinforcement learning model includes currently available rescue resources, task urgency and geographic location information, and the action space includes rescue personnel selection and path planning decisions; based on the state space and action space, a reward function is designed, and the reward function takes into account task completion time, resource utilization and rescue success rate; The third unit is used to construct a multi-objective optimization module including time constraints, space constraints and resource constraints. The multi-objective optimization module optimizes the task allocation scheme based on the improved particle swarm algorithm to obtain the optimal task allocation scheme; based on the optimal task allocation scheme, a task execution scheme is generated; The optimal task allocation scheme is sent to a real-time monitoring module, which monitors the task execution process and collects feedback information; The feedback information is input into the deep reinforcement learning model for online learning and model updating to achieve continuous optimization of the task allocation plan.

9. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method described in any one of claims 1 to 7.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images