Intelligent elevator precise control method and system based on multiple modes

Abstract

The invention relates to the technical field of elevator control, in particular to an intelligent elevator precise control method and system based on multiple modes, and the method comprises the steps that sensing monitoring data of all parts in an elevator system, monitoring video data of all floors and elevator button pressing monitoring data are obtained; the multi-branch feature extraction module based on the convolutional neural network is used for extracting multi-modal features from the standardized sensing monitoring data, the standardized monitoring video data and the standardized elevator button pressing monitoring data; multi-modal features with attention enhancement are obtained; taking the attention-enhanced multi-modal features as input of a gating circulation unit model, and predicting passenger flow states of all floors; according to the predicted floor passenger flow state and the probability thereof, an elevator control instruction corresponding to the prediction moment is generated; and the elevator control instruction is output to control operation of the elevator, the elevator dispatching strategy is optimized, dynamic adjustment can be achieved according to the real-time prediction result, and the operation efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of elevator control, and more specifically, to a multi-modal intelligent elevator precision control method and system. Background Art

[0002] In modern urban high-rise buildings and commercial complexes, elevators are an important means of vertical transportation. Their operating efficiency and user experience directly affect the function of the building and the satisfaction of users. With the rapid advancement of urbanization and the continuous increase in building heights, the limitations of traditional elevator control systems are becoming increasingly apparent and cannot fully meet the increasingly complex usage needs.

[0003] Existing elevator control methods usually rely on a single data source, such as button input or simple physical state data provided by elevator sensors. The main feature of these methods is a rule-based scheduling algorithm that determines the elevator's running path and parking order through preset logical judgments. However, this method has the following obvious technical problems:

[0004] First, existing methods lack the ability to perceive passenger flow dynamics in real time. Traditional elevator control is mainly based on simple logical inferences based on static data, which cannot fully reflect the complex changes in passenger flow during peak hours, off-peak hours, and emergencies. This lack of information directly leads to a decrease in elevator operating efficiency, especially in scenarios with dense passenger flow such as morning and evening peaks, where passengers often wait too long or elevators run empty.

[0005] Secondly, existing technologies have weak perception and data fusion capabilities for the environment inside and outside the elevator. Although some systems have introduced video surveillance or IoT sensors, these data are usually processed in isolation and cannot effectively fuse multimodal information to build a comprehensive operating status model. For example, a single button input can only reflect the basic needs of passengers, but cannot predict the actual passenger distribution or flow trends, and sensor data or video data cannot reveal the complex correlation of the overall operating status of the elevator system in a single analysis.

[0006] Thirdly, the existing elevator dispatching algorithms lack intelligence and dynamic adaptability. Traditional methods rely on pre-set fixed logic rules or simple statistical models, and cannot dynamically adjust dispatching strategies according to real-time passenger flow changes. This leads to inefficient elevator resource allocation, such as the inability to prioritize the needs of key floors when the floors are unevenly distributed, or the failure to allocate sufficient capacity in time in areas with dense passenger flow.

[0007] In addition, existing methods have great limitations in feature extraction and prediction algorithm design. It is difficult to fully capture the complex operating status of the elevator system with single modal feature extraction, and the traditional statistical prediction model has weak processing capabilities for time series characteristics and nonlinear relationships, making it difficult to achieve high-precision prediction of passenger flow status. The lack of such algorithmic capabilities directly affects the scientificity and accuracy of control instructions. Summary of the invention

[0008] In this context, the present invention proposes a multi-modal intelligent elevator precision control method and system, which overcomes the problems of traditional elevator control systems such as single perception dimension, insufficient feature processing capability and rigid scheduling strategy by fusing sensor data, video data and button input data.

[0009] The present invention provides a multi-modal intelligent elevator precise control method, comprising:

[0010] The acquisition steps include:

[0011] Obtain sensor monitoring data of each component in the elevator system, surveillance video data of each floor, and elevator button pressing monitoring data;

[0012] Preprocessing the data, including averaging and sparse coding, to obtain standardized sensor monitoring data, standardized surveillance video data, and standardized elevator button pressing monitoring data;

[0013] Processing steps include:

[0014] A multi-branch feature extraction module based on a convolutional neural network extracts multimodal features from standardized sensor monitoring data, standardized surveillance video data, and standardized elevator button pressing monitoring data;

[0015] Setting a corresponding attention matrix for the multimodal feature, and enhancing the feature by a Hadamard product operation to obtain an attention-enhanced multimodal feature;

[0016] The attention-enhanced multimodal features are used as the input of the gated recurrent unit model to predict the passenger flow status of all floors;

[0017] Output steps include:

[0018] Generate elevator control instructions corresponding to the predicted time according to the predicted floor passenger flow status and its probability;

[0019] The elevator control command is output to control the operation of the elevator.

[0020] Preferably, the obtaining step specifically includes:

[0021] Install sensors on elevator machine room components, elevator shaft components, elevator car components, elevator floor components, and elevator hall components to collect sensor monitoring data;

[0022] Set up monitoring areas on each floor of the elevator system and install cameras to collect monitoring video data;

[0023] Install sensors on elevator buttons to obtain elevator button press monitoring data.

[0024] Preferably, the processing steps specifically include:

[0025] Construct a multi-branch feature extraction module based on convolutional neural network to extract features from standardized sensor monitoring data, standardized surveillance video data, and standardized elevator button pressing monitoring data;

[0026] The extracted modal features are spliced ​​into multimodal features to further enhance the correlation between modalities.

[0027] Preferably, predicting the floor passenger flow state by using the gated recurrent unit model in the processing step specifically includes:

[0028] The multimodal features are used as the input of the gated recurrent unit model, and the sensor-monitored floor features and elevator control features are combined to comprehensively predict the passenger flow status of all floors.

[0029] Preferably, the gated recurrent unit model prediction process further includes:

[0030] Divide the passenger flow state sequence of each floor into training set and test set, and optimize the training model based on cross entropy loss;

[0031] Pearson correlation analysis was used to verify the prediction results in the test set to adjust the prediction model parameters.

[0032] Preferably, the output step specifically includes:

[0033] Based on the passenger flow state and the predicted probability at the prediction time, the Hadamard product operation is performed to obtain the similarity characteristics between the passenger flow state and the predicted probability;

[0034] Compare the similarity features with the preset passenger flow threshold. If it is lower than the threshold, generate an empty load control instruction, otherwise adjust the floor priority according to the predicted probability.

[0035] As a preference, it further comprises:

[0036] Dynamically switch the elevator operation mode based on real-time sensor monitoring data and passenger flow status;

[0037] The switching modes include full load priority mode, no load priority mode and energy saving priority mode.

[0038] Preferably, the attention enhancement module further comprises:

[0039] Use multi-head attention mechanism to capture the correlation between different modal features;

[0040] The consistency of multimodal feature fusion is optimized through the modal alignment loss function.

[0041] Preferably, the method further comprises:

[0042] Based on historical data analysis and current real-time predictions, a multi-objective optimization strategy is generated to comprehensively optimize elevator energy consumption, operating efficiency and user satisfaction.

[0043] A multi-modal intelligent elevator precision control system for executing the method includes:

[0044] Data acquisition terminal, used to obtain sensor monitoring data of each component in the elevator system, surveillance video data of each floor, and elevator button pressing monitoring data;

[0045] A multimodal deep feature extraction module for extracting sensor monitoring features, surveillance video features, and elevator button pressing features from the data;

[0046] Feature attention enhancement module, used to enhance multimodal features through attention matrix;

[0047] The floor passenger flow prediction module is used to predict the passenger flow status of all floors based on the gated recurrent unit model;

[0048] The elevator intelligent control module is used to generate elevator operation control instructions based on the predicted floor passenger flow status.

[0049] The core objective of the present invention is to solve the following technical problems:

[0050] 1. Improve the ability to perceive the elevator operating environment and passenger flow dynamics, and fully reflect the comprehensive status inside and outside the elevator system;

[0051] 2. Through multimodal data fusion and deep learning algorithms, high-precision dynamic prediction of floor passenger flow status can be achieved;

[0052] 3. Optimize the elevator dispatching strategy so that it can be dynamically adjusted according to real-time prediction results, improve operating efficiency, reduce energy consumption, and improve user satisfaction.

[0053] The present invention has the following beneficial effects:

[0054] The present invention achieves a breakthrough in perception dimension and data processing capability through multimodal data fusion technology, overcoming the limitations of single-modal data processing in existing methods. Specifically, the present invention uses sensor monitoring data to capture the time series characteristics of the physical state of the elevator, uses video data to reflect the distribution and dynamic changes of passenger flow on each floor, and combines button press data to supplement user direct demand information. These data are processed by a multi-branch convolutional neural network to generate high-dimensional multimodal features, and then the attention enhancement mechanism is used to give the features adaptive weight allocation capabilities to achieve priority extraction of key features.

[0055] Another notable feature of the present invention is the introduction of a gated recurrent unit model to perform time series modeling of multimodal features. The model effectively captures the time series characteristics and nonlinear change relationship of passenger flow status through a dynamic memory unit, thereby achieving high-precision prediction of floor passenger flow status. By generating dynamic scheduling instructions through prediction results, the present invention can effectively avoid the problems of elevator no-load operation and uneven resource allocation, greatly improving the overall efficiency of elevator operation.

[0056] In addition, the method of the present invention also achieves multi-dimensional improvement of control instructions through a multi-objective optimization strategy. According to real-time passenger flow forecasts and historical data analysis, the elevator energy consumption, operating efficiency and user satisfaction are comprehensively optimized to ensure the flexible adaptability of the system in different scenarios. For example, during peak hours, priority is given to responding to dense passenger flow needs, and during non-peak hours, the operating strategy is adjusted with energy saving as the goal. This dynamic adjustment capability is not achievable by existing methods.

[0057] In summary, the present invention solves key technical problems in traditional elevator control systems, such as insufficient perception, single data processing, and rigid scheduling strategies, through deep collaboration between technical features. Its innovation is reflected in the improvement of the entire chain of data fusion, algorithm design, and control strategy optimization, which significantly improves the intelligence level and operating efficiency of the elevator system and has a wide range of practical application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 The figure is a flow chart of the method of the present invention.

[0059] Figure 2 This is a logic block diagram of data collection and preprocessing of the present invention.

[0060] Figure 3 This is a logic block diagram of the multimodal feature extraction of the present invention.

[0061] Figure 4 This is a logic block diagram of feature attention enhancement of the present invention.

[0062] Figure 5 It is the logic block diagram of floor passenger flow prediction of the present invention.

[0063] Figure 6 This is a logic block diagram of the elevator intelligent control of the present invention. DETAILED DESCRIPTION

[0064] Please refer to Figure 1-6 The present invention provides a multi-modal intelligent elevator precision control method, comprising:

[0065] The acquisition steps include:

[0066] Obtain sensor monitoring data of each component in the elevator system, surveillance video data of each floor, and elevator button pressing monitoring data;

[0067] The data is preprocessed, including averaging and sparse coding, to obtain standardized sensor monitoring data, standardized surveillance video data, and standardized elevator button pressing monitoring data.

[0068] Processing steps include:

[0069] A multi-branch feature extraction module based on a convolutional neural network extracts multimodal features from standardized sensor monitoring data, standardized surveillance video data, and standardized elevator button pressing monitoring data;

[0070] Setting a corresponding attention matrix for the multimodal feature, and enhancing the feature by a Hadamard product operation to obtain an attention-enhanced multimodal feature;

[0071] The attention-enhanced multimodal features are used as input to the gated recurrent unit model to predict the passenger flow status of all floors.

[0072] Output steps include:

[0073] Generate elevator control instructions corresponding to the predicted time according to the predicted floor passenger flow status and its probability;

[0074] The elevator control command is output to control the operation of the elevator.

[0075] In the method of the present invention, data acquisition is the key starting point. For example, the sensor monitoring data includes the vibration characteristics of the elevator shaft, the speed and position signals of the car, which are obtained through multiple high-precision acceleration sensors installed in the shaft, machine room and car.

[0076] The data averaging process uses a sliding window method with a window length of 5 seconds to smooth out random noise.

[0077] Data sparse coding is achieved through L1 regularization to retain key features and improve the efficiency of algorithm training.

[0078] In the processing step, the multi-branch convolutional neural network is designed with three branches to process sensor data, video data, and button data respectively. The specific convolution kernel size is preferably 3×3, and combined with the ReLU activation function, it can effectively capture spatial and temporal features. Feature attention enhancement is achieved through the following formula:

[0079] F ′ =A⊙F

[0080] Among them, F is the original feature matrix, A is the attention matrix, and ⊙ is the Hadamard product operator. Here, the attention matrix A is generated by the multi-head attention mechanism to increase the weight of the key area features.

[0081] In the prediction process, the input of the gated recurrent unit model is the enhanced multimodal features, which are combined with the historical passenger flow sequence to dynamically predict the passenger flow status of the floor. For example, a congestion signal is generated when the predicted number of people on the floor is 10±2. The output step generates control instructions based on the prediction results, such as giving priority to dispatching empty elevators to peak floors.

[0082] In one embodiment of the present invention, the obtaining step specifically includes:

[0083] Install sensors on elevator machine room components, elevator shaft components, elevator car components, elevator floor components, and elevator hall components to collect sensor monitoring data;

[0084] Set up monitoring areas on each floor of the elevator system and install cameras to collect monitoring video data;

[0085] Install sensors on elevator buttons to obtain elevator button press monitoring data.

[0086] Preferably, in one embodiment of the present invention, the sensor uses a high-precision MEMS accelerometer to collect the vibration and tilt angle of the elevator running state in real time. For example, the sensor installed in the hoistway can upload data to the central control system through the Bluetooth 5.0 network node to ensure low power consumption and high real-time performance.

[0087] The camera module is preferably a 2-megapixel wide-angle lens that can cover the entire area of ​​the entrance to each floor. Its video data is compressed by H.264 encoding and uploaded to reduce the transmission bandwidth requirement. The elevator button sensor uses a capacitive touch sensor to achieve high-sensitivity data collection by detecting touch pressure. For example, when the button pressing force exceeds 0.5N, it is recorded as a valid input.

[0088] The collection and real-time transmission of these data provide high-quality input for subsequent multimodal processing.

[0089] In one embodiment of the present invention, the processing steps specifically include:

[0090] Construct a multi-branch feature extraction module based on convolutional neural network to extract features from standardized sensor monitoring data, standardized surveillance video data, and standardized elevator button pressing monitoring data;

[0091] The extracted modal features are spliced ​​into multimodal features to further enhance the correlation between modalities.

[0092] Preferably, the multi-branch feature extraction module of the present invention is designed as follows:

[0093] The first branch is used to process sensor data and adopts a deep convolutional network to capture time series features.

[0094] The second branch is used to process video data and use the ResNet architecture to extract the spatial features of personnel distribution. For example, the convolutional layer is used to extract the heat map of personnel distribution at the floor entrance.

[0095] The third branch is used to process button data and adopts a lightweight convolutional network to quickly capture the characteristics of button pressing patterns.

[0096] These features are concatenated into a multimodal feature vector using the following formula:

[0097] F multi =[F sensor ,F video ,F button ]

[0098] Among them, F multi is a multimodal feature vector, which combines the features of three modes; F sensor is the sensing feature, indicating the operating status; F video is the video feature, indicating the distribution of people on each floor; button It is a button feature, indicating user operation behavior.

[0099] The spliced ​​multimodal features are further fed into the attention mechanism module to provide a comprehensive information basis for the subsequent prediction model. Through multimodal fusion, the system can comprehensively analyze the dynamic state inside and outside the elevator and improve the accuracy of decision-making.

[0100] In one embodiment of the present invention, predicting the floor passenger flow state by using the gated recurrent unit model in the processing step specifically includes:

[0101] The multimodal features are used as the input of the gated recurrent unit model, and the sensor-monitored floor features and elevator control features are combined to comprehensively predict the passenger flow status of all floors.

[0102] Preferably, in the method of the present invention, the gated recurrent unit model is used to process the time series characteristics of the multimodal features and to perform passenger flow prediction in combination with historical data. The input of the gated recurrent unit model includes multimodal features, sensor-monitored floor features, and elevator control features. The sensor-monitored floor features represent the real-time status of each floor, such as vibration frequency and temperature distribution; the elevator control features represent the current operation mode of the elevator, such as the priority of floor stops.

[0103] The gated recurrent unit model describes its state update process through the following formula:

[0104]

[0105] z t =σ(W z ·[h t-1 ,x t ]+b z )

[0106] r t =σ(W r ·[h t-1 ,x t ]+b r )

[0107] Where: h t is the hidden state of the current time step, representing the characteristic expression of the prediction result, h t-1 is the hidden state of the previous time step; is a candidate hidden state; x t is the input feature of the current time step; z t is the update gate, which controls the update ratio of the state; r t To reset the gate, control the degree of forgetting; W h ,W z ,W r is the weight matrix of the model, b h ,b z ,b r is the bias term; ⊙ is the Hadamard product, which represents the element-wise product operation; σ is the Sigmoid activation function, and tanh is the hyperbolic tangent function.

[0108] In a preferred embodiment of the present invention, the results of passenger flow state prediction include the density of people, flow trend and dwell time of each floor. For example, through historical data analysis, the predicted passenger flow of a certain floor during the morning rush hour on weekdays may reach 15±3 people / minute. This method significantly improves the prediction accuracy through deep modeling of multimodal features, providing reliable data support for subsequent control strategies.

[0109] In a preferred embodiment of the present invention, the gated cycle unit model prediction process further includes:

[0110] Divide the passenger flow state sequence of each floor into training set and test set, and optimize the training model based on cross entropy loss;

[0111] Pearson correlation analysis was used to verify the prediction results in the test set to adjust the prediction model parameters.

[0112] Preferably, the method of the present invention optimizes the prediction model in a data-driven manner. In the model training stage, the floor passenger flow state sequence is divided into a training set and a test set. The training set is used to learn the model parameters, and the test set is used to verify the prediction performance.

[0113] The cross entropy loss function is used to measure the difference between the model's predicted value and the actual value, and is defined as follows:

[0114]

[0115] Among them, L is the cross entropy loss value; N is the number of samples; y i is the actual passenger flow status; is the passenger flow state probability predicted by the model.

[0116] In a preferred embodiment, an early stopping strategy is used when training the model. When the loss value of the test set does not improve within 10 consecutive iterations, training is stopped to prevent overfitting.

[0117] In addition, the Pearson correlation analysis formula is used to verify the correlation between the predicted results and the actual data:

[0118]

[0119] Where r is the Pearson correlation coefficient; x i ,y i are the predicted value and the actual value respectively; are the means of the predicted and actual values, respectively.

[0120] When the r value is greater than 0.85, it indicates that the prediction result has a high consistency with the actual data. The method of the present invention optimizes the prediction performance of the model through the above steps and ensures its robustness in diverse scenarios.

[0121] In a preferred embodiment, the output step specifically includes:

[0122] Based on the passenger flow state and the predicted probability at the prediction time, the Hadamard product operation is performed to obtain the similarity characteristics between the passenger flow state and the predicted probability;

[0123] Compare the similarity features with the preset passenger flow threshold. If it is lower than the threshold, generate an empty load control instruction, otherwise adjust the floor priority according to the predicted probability.

[0124] Preferably, the method of the present invention utilizes Hadamard product operation to combine passenger flow status and predicted probability to generate similarity features. The specific calculation formula is:

[0125] S=P⊙C

[0126] Among them, S is the similarity feature; P is the prediction probability matrix; C is the passenger flow status matrix.

[0127] When the value of S is lower than the set passenger flow threshold (e.g., 0.2), the system determines that the passenger flow on the current floor is sparse, and thus generates an idle operation instruction first to save energy. For example, during the non-peak period at night, the similarity feature value of a certain floor is only 0.15, and the system directly skips the floor according to this method.

[0128] When the value of S is higher than the threshold, the service priority of the floor is adjusted according to the predicted probability. For example, during peak hours, if the similarity feature value of a floor is 0.9, the system will prioritize arranging the elevator to run to this floor at full load to improve operating efficiency.

[0129] In a preferred embodiment, it further comprises:

[0130] Dynamically switch the elevator operation mode based on real-time sensor monitoring data and passenger flow status;

[0131] The switching modes include full load priority mode, no load priority mode and energy saving priority mode.

[0132] In a preferred embodiment of the present invention, the elevator operation mode is dynamically adjusted according to the real-time collected sensor monitoring data and passenger flow status. For example, during the morning peak period, when the passenger flow of a certain floor is predicted to exceed 30 people, it is switched to the full load priority mode, and more empty elevators are arranged to serve the peak floors first; and during the non-peak period at night, when the passenger flow of the floor is less than 5 people, it is switched to the energy-saving priority mode to reduce empty-load operation.

[0133] The switching strategy of the operating mode is based on the following rules:

[0134] Full load priority mode: maximize elevator capacity and reduce waiting time.

[0135] No-load priority mode: quickly respond to specific floor needs, such as VIP service scenarios.

[0136] Energy-saving priority mode: Reduces system power consumption and extends device life by reducing the number of starts and stops.

[0137] Through these dynamic adjustments, the method of the present invention can flexibly adapt to the needs of different scenarios and significantly improve the operating efficiency and user satisfaction of the elevator system.

[0138] In a preferred embodiment of the present invention, the attention enhancement module further comprises:

[0139] Use multi-head attention mechanism to capture the correlation between different modal features;

[0140] The consistency of multimodal feature fusion is optimized through the modal alignment loss function.

[0141] Preferably, the method of the present invention deeply enhances multimodal features through a multi-head attention mechanism. The multi-head attention mechanism can capture the potential relationship between sensor features, video features and button features, and its calculation process is as follows:

[0142]

[0143] Among them, Q is the query matrix (Query), which is used to extract relevant information; K is the key matrix (Key), which indicates the importance of the feature; V is the value matrix (Value), which contains the original feature information; d k is the dimension of the key matrix, which is used for normalization calculation. The multi-head attention mechanism captures the diversity of multimodal features by computing multiple independent attention heads in parallel:

[0144] MultiHead(Q,K,V)=[head1,head2,…,head h ]W O

[0145] Among them, head i represents the output of the i-th attention head; W O is a linear transformation matrix, and the outputs of all attention heads are integrated into the final multimodal features. In order to optimize the fusion consistency of different modal features, the method of the present invention introduces a modal alignment loss function, which is defined as:

[0146]

[0147] Among them, L align is the modality alignment loss value; F sensor ,F video ,F button are the embedding vectors of sensor, video and button features respectively; N is the number of samples.

[0148] Preferably, in the embodiment of the present invention, the heterogeneity problem in multimodal feature fusion is effectively solved by combining the multi-head attention mechanism and the modal alignment loss function, and the integrity and consistency of feature representation are improved. For example, in practical applications, the fusion accuracy of multimodal features is improved by more than 10%, providing more accurate input for subsequent passenger flow prediction.

[0149] In a preferred embodiment of the present invention, the method further comprises:

[0150] Based on historical data analysis and current real-time predictions, a multi-objective optimization strategy is generated to comprehensively optimize elevator energy consumption, operating efficiency and user satisfaction.

[0151] Preferably, the method of the present invention combines historical data and real-time prediction results to dynamically adjust the elevator operation mode through a multi-objective optimization strategy. Specifically, the objective function of the multi-objective optimization strategy is defined as:

[0152] MinimizeZ=w1E+w2T+w3S

[0153] Among them, Z is the comprehensive optimization target; E is the energy consumption of elevator operation; T is the passenger waiting time; S is the negative indicator of user satisfaction; w1, w2, w3 are weight parameters, indicating the priority among the three targets.

[0154] Preferably, the weight parameter is set dynamically according to the specific scenario. For example, during peak hours, w2 has a higher weight, mainly to reduce the waiting time of passengers; during night time, w1 has a higher weight, mainly to save energy.

[0155] In the embodiment of the present invention, through the multi-objective optimization strategy, the system can be flexibly adjusted in the following scenarios:

[0156] Scenario 1: Office building peak period

[0157] Historical data analysis shows that the passenger flow on a certain floor surges between 8:00 and 9:00 a.m. every day. By increasing the weight of (w_2), the system prioritizes unloaded elevators to peak floors.

[0158] Scenario 2: Shopping mall energy-saving mode

[0159] After 22:00 at night, the number of customers in the mall decreases significantly. The system aims to reduce energy consumption and achieves energy saving by reducing unnecessary stops, saving up to 15% of electricity.

[0160] In a preferred embodiment of the present invention, the present invention further discloses a multi-modal intelligent elevator precision control system, comprising:

[0161] Data acquisition terminal 1, used to obtain sensor monitoring data of various components in the elevator system, monitoring video data of each floor, and elevator button pressing monitoring data;

[0162] A multimodal deep feature extraction module 2, used to extract sensor monitoring features, surveillance video features and elevator button pressing features from the data;

[0163] Feature attention enhancement module 3, used to enhance multimodal features through attention matrix;

[0164] The floor passenger flow prediction module 4 is used to predict the passenger flow status of all floors based on the gated cycle unit model;

[0165] The elevator intelligent control module 5 is used to generate elevator operation control instructions according to the predicted floor passenger flow status.

[0166] The system of the present invention is composed of multiple modules, and each module works together to realize intelligent control of the elevator. Preferably, in the embodiment of the present invention, the function of each module is described as follows:

[0167] The data acquisition terminal 1 includes sensors, cameras and button modules, and transmits data in real time through the Internet of Things network. For example, the vibration sensor installed in the elevator car can record acceleration data at a frequency of 100 times per second to ensure that the accurate operating status is captured.

[0168] The multimodal deep feature extraction module 2 extracts high-dimensional features of various types of data through convolutional neural networks, such as the time series pattern of sensor features, the personnel distribution map of video features, and the pressing behavior pattern of button features.

[0169] The feature attention enhancement module 3 dynamically adjusts the feature weights through a multi-head attention mechanism to capture the key correlations between the features of each modality. For example, during peak hours, the weight of surveillance video features may be higher to prioritize responding to crowd gatherings.

[0170] The floor passenger flow prediction module 4 performs time series modeling on multimodal features through a gated recurrent unit model to predict passenger flow and trends on each floor in real time. Preferably, the module can dynamically adjust the prediction accuracy in combination with historical data.

[0171] The elevator intelligent control module 5 generates specific control instructions according to the prediction results, such as arranging unloaded elevators to serve peak floors first or adjusting the elevator operation mode to achieve energy saving.

[0172] Through the synergy of various modules, this system can adapt to the needs of different scenarios and effectively improve elevator operation efficiency and user satisfaction. For example, in experimental tests, this system reduced the average passenger waiting time by more than 20% and reduced energy consumption by 15%.

[0173] It should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A multi-modal intelligent elevator precision control method, characterized in that: include: The acquisition steps include: Obtain sensor monitoring data of each component in the elevator system, surveillance video data of each floor, and elevator button pressing monitoring data; Preprocessing the data, including averaging and sparse coding, to obtain standardized sensor monitoring data, standardized surveillance video data, and standardized elevator button pressing monitoring data; Processing steps include: A multi-branch feature extraction module based on a convolutional neural network extracts multimodal features from standardized sensor monitoring data, standardized surveillance video data, and standardized elevator button pressing monitoring data; Setting a corresponding attention matrix for the multimodal feature, and enhancing the feature by a Hadamard product operation to obtain an attention-enhanced multimodal feature; The attention-enhanced multimodal features are used as the input of the gated recurrent unit model to predict the passenger flow status of all floors; Output steps include: Generate elevator control instructions corresponding to the predicted time according to the predicted floor passenger flow status and its probability; The elevator control command is output to control the operation of the elevator.

2. According to claim 1, a multi-modal intelligent elevator precision control method is characterized in that: The acquisition step specifically includes: Install sensors on elevator machine room components, elevator shaft components, elevator car components, elevator floor components, and elevator hall components to collect sensor monitoring data; Set up monitoring areas on each floor of the elevator system and install cameras to collect monitoring video data; Install sensors on elevator buttons to obtain elevator button press monitoring data.

3. According to the multi-modal intelligent elevator precise control method of claim 1, it is characterized in that: The processing steps specifically include: Construct a multi-branch feature extraction module based on convolutional neural network to extract features from standardized sensor monitoring data, standardized surveillance video data, and standardized elevator button pressing monitoring data; The extracted modal features are spliced ​​into multimodal features to further enhance the correlation between modalities.

4. The multi-modal intelligent elevator precise control method according to claim 1, characterized in that: The processing step of predicting the floor passenger flow state by using the gated recurrent unit model specifically includes: The multimodal features are used as the input of the gated recurrent unit model, and the sensor-monitored floor features and elevator control features are combined to comprehensively predict the passenger flow status of all floors.

5. The multi-modal intelligent elevator precise control method according to claim 4, characterized in that: The gated recurrent unit model prediction process also includes: Divide the passenger flow state sequence of each floor into training set and test set, and optimize the training model based on cross entropy loss; Pearson correlation analysis was used to verify the prediction results in the test set to adjust the prediction model parameters.

6. The multi-modal intelligent elevator precise control method according to claim 1, characterized in that: The output step specifically includes: Based on the passenger flow state and the predicted probability at the prediction time, the Hadamard product operation is performed to obtain the similarity characteristics between the passenger flow state and the predicted probability; Compare the similarity features with the preset passenger flow threshold. If it is lower than the threshold, generate an empty load control instruction, otherwise adjust the floor priority according to the predicted probability.

7. The multi-modal intelligent elevator precise control method according to claim 1, characterized in that: Further including: Dynamically switch the elevator operation mode based on real-time sensor monitoring data and passenger flow status; The switching modes include full load priority mode, no load priority mode and energy saving priority mode.

8. The multi-modal intelligent elevator precise control method according to claim 1, characterized in that: The attention enhancement module further comprises: Use multi-head attention mechanism to capture the correlation between different modal features; The consistency of multimodal feature fusion is optimized through the modal alignment loss function.

9. The multi-modal intelligent elevator precise control method according to claim 1, characterized in that: The method further comprises: Based on historical data analysis and current real-time predictions, a multi-objective optimization strategy is generated to comprehensively optimize elevator energy consumption, operating efficiency and user satisfaction.

10. A multi-modal intelligent elevator precision control system implementing the method according to any one of claims 1 to 9, characterized in that: include: Data acquisition terminal, used to obtain sensor monitoring data of each component in the elevator system, surveillance video data of each floor, and elevator button pressing monitoring data; A multimodal deep feature extraction module for extracting sensor monitoring features, surveillance video features, and elevator button pressing features from the data; Feature attention enhancement module, used to enhance multimodal features through attention matrix; The floor passenger flow prediction module is used to predict the passenger flow status of all floors based on the gated recurrent unit model; The elevator intelligent control module is used to generate elevator operation control instructions based on the predicted floor passenger flow status.

Images