Target detection high-precision elevator traffic mode recognition method
By acquiring elevator passenger flow information through the improved YOLOX-Swin Transformer object detection algorithm and combining it with a traffic flow model, the timeliness problem of traffic pattern recognition in elevator group control systems is solved, and high-precision elevator scheduling optimization is achieved.
Patent Information
- Application Number
- CN202310581451.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-23
- Publication Date
- 2025-11-25
- Estimated Expiration
- 2043-05-23
AI Technical Summary
In existing elevator group control systems, traffic pattern recognition models based on time series lack timeliness and cannot effectively cope with continuously changing passenger flow conditions, resulting in excessively long waiting and riding times for passengers, which affects passenger comfort and system efficiency.
An improved YOLOX-Swin Transformer object detection algorithm is adopted. Passenger distribution information is obtained through an object detection network. Combined with a traffic flow model, the current passenger flow situation is identified in real time and an appropriate traffic mode is selected.
It improves the accuracy and timeliness of elevator traffic pattern recognition, optimizes elevator scheduling, reduces passenger waiting and riding time, and enhances passenger comfort and system performance.
Smart Images
Figure CN116824482B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to an elevator traffic pattern recognition method, and more particularly to a high-precision elevator traffic pattern recognition method with target detection. Technical Background
[0002] Elevators have evolved over 100 years from early mechanical control methods to intelligent control. The combination of microprocessor control systems and variable frequency speed control technology has made elevator systems increasingly stable. Especially after the advent of PLCs, elevator control has gradually matured. With urbanization and the diversified development of buildings, buildings are becoming increasingly tall, and single elevator control systems can no longer meet passenger demands. Multi-elevator group control systems have gradually become the primary means of vertical transportation in high-rise buildings.
[0003] An elevator is a control system operating within a building. Its performance must meet the building's requirements, especially in multi-functional buildings. While fulfilling passenger scheduling needs, the elevator system must also conserve energy and avoid excessive resource waste. When scheduling, the group control system needs to consider the average waiting time and average riding time of passengers to prevent passenger frustration caused by excessive waiting or riding times. After passengers enter the elevator car, sufficient space must be provided to avoid overcrowding and reduced passenger comfort. In addition, factors such as maximum waiting time, maximum riding time, and elevator response time must be considered. Reasonable elevator scheduling can effectively improve the overall performance of the elevator system.
[0004] Elevator systems need to consider current passenger flow information during scheduling and select the appropriate traffic mode based on passenger flow distribution. Therefore, the accuracy of traffic pattern recognition directly affects scheduling performance. Traffic pattern recognition in group-controlled elevator systems mostly uses time-series algorithms, which leads to a lack of timeliness in time-series-based elevator traffic models. Furthermore, data can only be collected for buildings with specific passenger flows during modeling, making the model unable to cope with continuously changing passenger flow conditions.
[0005] Most traffic pattern recognition and passenger flow prediction models in elevator group control systems employ time series algorithms, resulting in a lack of timeliness in time-series-based elevator traffic models. Furthermore, modeling can only collect data for buildings with specific passenger flows, making the models unable to handle continuously changing passenger flow conditions. Summary of the Invention
[0006] The purpose of this invention is to propose a high-precision elevator traffic pattern recognition method based on object detection. This method uses an improved YOLOX-Swin Transformer object detection algorithm to obtain the distribution of passengers on each floor, thus obtaining the real passenger flow situation at the current moment. The data is then transmitted to a traffic flow model, and the system ultimately selects the appropriate traffic mode based on the passenger flow data. Compared with traditional time series models, object detection technology is more efficient, accurate, and real-time. An elevator system based on object detection gains a "visual" function, enabling it to better obtain passenger status information, thereby improving the accuracy of traffic pattern recognition and ultimately optimizing elevator dispatching.
[0007] The present invention adopts the following technical solution:
[0008] A high-precision elevator traffic pattern recognition method for target detection is disclosed. The method constructs a network model for passenger traffic pattern recognition. The traffic pattern recognition module obtains real-time passenger flow information based on target detection algorithms and traffic flow model predictions, and selects the current traffic mode based on this information. Specifically, the method includes the following steps:
[0009] 1) Create the passenger dataset required for the elevator system and complete the dataset annotation;
[0010] 2) Configure the network environment using the deep learning PyTorch framework, and complete the construction of the YOLOX model within this environment;
[0011] 3) Improve YOLOX's backbone by using the self-attention mechanism Swing Transformer to enhance feature extraction;
[0012] 4) Improve the spatial pyramid pooling module of YOLOX by adopting the CSPnet structure and SPPF concept to enhance the depth and computational speed of the spatial pyramid pooling module;
[0013] 5) Improve YOLOX's Neck by using a Coordinate Attention mechanism to enhance feature fusion at different sizes;
[0014] 6) Construct a traffic flow model for the system and select the current traffic mode based on the passenger distribution obtained by the target detection module.
[0015] The method for high-precision elevator traffic pattern recognition using object detection involves processing image datasets from two system input models. For training the object detection network, a dataset of passenger images from real elevator environments is used, retaining the optimal weights for model training. Video recordings from monitoring equipment are collected and processed frame-by-frame to obtain passenger images in various elevator environments. First, the images are converted to JPG format based on dataset annotations. Then, the labelimg annotation tool is used to annotate passenger categories in all image datasets, and the labels are output in XML format. When inputting the data into the network, the XML file recording the annotation information is input into the object detection model in TXT format. During network training, the data needs to be divided into training and test sets; here, 90% of the dataset is used as the training set, and the remaining 10% is used as the test set.
[0016] The method for high-precision elevator traffic pattern recognition in target detection involves processing the image dataset input to the network to build a target detection model. The model architecture consists of four parts: Input, Backbone, Neck, and Head. The Input layer preprocesses the input images using Mosaic and Mixup data augmentation techniques to enhance dataset diversity and model robustness. The Backbone employs the CSPDarknet model, adding a CSPnet structure to Darknet53. This structure splits the input feature map into two parts: the backbone is divided into the original residual structure and subjected to a series of convolutional operations, while the branch parts are left unprocessed. The two branches are then concatenated, and the fused features are stacked. The Neck uses a PAnet path aggregation network, consisting of a bottom-up upsampling tower and a top-down downsampling tower. These towers upsample features of different sizes, concatenating them when their sizes are equal. Convolutional layers fuse these features to extract information from features of different sizes. A decoupled network is used. The head uses three sizes of features from the feature fusion network for prediction, and improves model accuracy by using Anchor-free and SimOTA-based methods.
[0017] The aforementioned high-precision elevator traffic pattern recognition method for target detection involves building a target detection model, using the Swing Transformer to optimize the model's backbone, and extracting feature maps with feature sizes after downsampling by 8x, 16x, and 32x. A sliding window is used for multi-window information exchange to improve the model's attention to passengers.
[0018] The aforementioned high-precision elevator traffic pattern recognition method for target detection improves upon Backbone by enhancing feature extraction through improvements to spatial pyramid pooling. CSPnet is used to expand the model's receptive field, and a CSP structure is introduced into SPP. This module features a residual edge to assist in optimization and feature extraction, increasing the model's depth. Based on the SPPF concept, the pooling structure of SPP is optimized, improving computational speed without changing the number of parameters.
[0019] The aforementioned high-precision elevator traffic pattern recognition method for target detection improves upon spatial pyramid pooling. To enhance passenger information across multiple sizes during the feature fusion stage, a CA attention mechanism is employed to strengthen the features after fusion at different sizes. This CA attention mechanism is added to the improved YOLOX-Swin Transformer network, embedding positional information into channel attention to obtain attention weights on the feature map in height and width. Finally, by incorporating these attention weights into the original feature map, the network's detection accuracy is enhanced. The formula for the CA attention mechanism is as follows:
[0020] (1)
[0021] The above model optimization was completed. Compared with the original YOLOX object detection algorithm, the map value increased by 5%, which is of great significance for the model to detect passenger flow.
[0022] The method described above is a high-precision elevator traffic pattern recognition method for target detection. This method involves various optimizations to the model, training and testing the performance of the target detection model to achieve traffic pattern recognition. The method sets parameters such as the number of data points input into the network at one time, the number of training rounds, and the number of worker threads to begin model training. After training, the method checks the model's performance metrics for reasonableness, activates a camera to detect the input data, and checks the confidence level of the predicted bounding boxes. The elevator system uses the target detection network to extract passenger data, recording the number of people waiting on each floor and the number of passengers in the elevator car. At certain intervals, the system records data such as average passenger travel time, average waiting time, and dwell time on the target floor, and transmits this data to the information management module for elevator scheduling optimization. The traffic pattern recognition module obtains the pedestrian flow data from the target detection and selects the appropriate traffic mode based on the real-time pedestrian flow data.
[0023] The present invention has the following beneficial effects:
[0024] This invention employs an object detection-based method, training a network model using a large dataset of passenger images to construct the system's passenger recognition model. During system operation, the object detection module detects passengers in the elevator lobby and inside the elevator car, acquiring passenger distribution information and obtaining real-time passenger flow data. Compared to traditional time-series prediction algorithms, this method offers significantly improved timeliness. By transmitting actual passenger flow data to the traffic pattern recognition module, the most suitable traffic pattern for the current passenger flow can be determined. Furthermore, data such as average elevator travel time and average waiting time can be used to optimize the scheduling algorithm. Improvements have been made to the YOLOX object detection algorithm, enhancing network accuracy and contributing to the overall accuracy of traffic pattern recognition in the system. Attached Figure Description
[0025] Figure 1 This is a functional structure diagram of an elevator group control system;
[0026] Figure 2 This is a diagram of the improved YOLOX-Swin Transformer model architecture;
[0027] Figure 3 This is a diagram of the CA attention mechanism network structure;
[0028] Figure 4 This is a flowchart of the model training process;
[0029] Figure 5 It is a training accuracy map;
[0030] Figure 6 This is a graph of the loss function. Detailed Implementation
[0031] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.
[0032] This invention employs an improved YOLOX-Swin Transformer object detection algorithm to acquire images of passengers in an elevator, extracts passenger feature information from the images for prediction, and obtains the current distribution of passengers inside the elevator car and in the waiting hall. The object detection module transmits passenger status information to the traffic flow model, and the system selects a traffic mode based on the real passenger flow information at the current moment.
[0033] This invention applies target detection to traffic pattern recognition in elevator group control systems. It mainly consists of two parts: a target detection algorithm network and traffic pattern recognition. The functional module structure diagram of the elevator system is shown below. Figure 1 As shown.
[0034] The object detection network requires preprocessing of the input data. First, the collected passenger image dataset is converted to JPG format. Then, the labelimg tool is used to label the categories detected by the network, recording the labeling information in an XML file. The data from the XML file is then input into the object detection algorithm network in TXT format. During training, the dataset is divided into training and testing sets in a 9:1 ratio. Each image in the dataset undergoes data augmentation before entering the input, and finally, its size is set to 640×640 pixels with 3 channels.
[0035] This project involves building the YOLOX-Swin Transformer object detection algorithm and configuring its runtime environment. PyCharm is used as the development tool, and the PyTorch library is used to configure the network's runtime environment. After completing the initial work, the parameters of the YOLOX-Swin Transformer object detection algorithm need to be set and the model built. The YOLOX-Swin Transformer model is as follows: Figure 2 As shown, the network framework mainly consists of an Input layer, a Backbone network, a Neck layer, and a Head output layer. The Input layer processes the input image, employing Mosaic and Mixup data augmentation techniques to improve dataset diversity and enhance model robustness. The Backbone uses a Swin Transformer to extract features from the input image, strengthening the network's passenger feature information through an Encoder-Decoder approach within a self-attention mechanism. The Neck layer uses an FPN+PAN structure for feature fusion and employs a CA attention mechanism to enhance the information after feature fusion at different sizes. The CA attention mechanism is described in detail below. Figure 3 As shown, the Head stage employs a SIOU loss calculation that considers vector angles to improve the accuracy of model predictions.
[0036] After the improved model is built, the preprocessed dataset needs to be input into the network for training. The training experiment uses a computer running Windows 10, with computing power provided by the AutoDL cloud platform, an NVIDIA GeForce GTX 3090 GPU, and PyTorch V1.10.0 and torchvision V 0.11.0 as the algorithm framework. Before training, the model parameters need to be modified, including setting the path, type, and number of training datasets; selecting hyperparameters and activation functions suitable for the dataset training type; and setting the training worker threads, the number of images input to the network, and the number of training epochs. After setting the model parameters, training begins to obtain model weights that conform to the system. The model training process is as follows: Figure 4 As shown.
[0037] The training progress is recorded using model training logs. Performance metrics such as Precision, Recall, F1 score, and map are used to evaluate model performance. Model accuracy is also assessed. Figure 5 As shown, the loss function is as follows Figure 6 As shown, real passenger elevator images are used as input to the model to test the accuracy of the model in recognizing passenger prediction boxes.
[0038] The elevator system uses object detection algorithms to obtain the current passenger distribution. Cameras on different floors, both in the waiting areas and inside the elevator car, detect the number and location of passengers. The system records the number of people waiting on each floor and the number of passengers riding in the elevator car. At regular intervals, it records data such as average riding time, average waiting time, and dwell time on the target floor, and transmits this data to the information management module for elevator scheduling optimization. The traffic pattern recognition module receives the pedestrian flow data from object detection and selects the appropriate traffic mode based on the real-time pedestrian flow data. By using object detection to identify elevator traffic patterns, real-time and effective pedestrian flow data can be detected. Example
[0039] 1. Create the passenger dataset required for the elevator system and complete the dataset annotation;
[0040] 2. Configure the network environment using the deep learning PyTorch framework, and complete the construction of the YOLOX model within this environment;
[0041] 3. Improve YOLOX's backbone by using the self-attention mechanism, the Swing Transformer, to enhance feature extraction;
[0042] 4. Improve the spatial pyramid pooling module of YOLOX by adopting the CSPnet structure and SPPF concept to enhance the depth and computation speed of the spatial pyramid pooling module.
[0043] 5. Improve YOLOX's Neck by using a Coordinate Attention mechanism to enhance feature fusion at different sizes;
[0044] 6. Test the target detection model, build the system's traffic flow model, and select the current traffic mode based on the passenger distribution obtained by the target detection module.
[0045] The specific implementation steps of step 1 are as follows:
[0046] Collect the passenger image dataset required for the elevator system and annotate the dataset using the labelimg annotation tool;
[0047] Output the labeled dataset in YOLO format (txt format);
[0048] The dataset is divided into training and test sets in a 9:1 ratio.
[0049] The specific implementation steps of step 2 are as follows:
[0050] Configure and import the runtime environment for the object detection model;
[0051] The YOLOX model is constructed, consisting of an Input layer, a Backbone network, a Neck network layer, and a Head output layer.
[0052] The specific implementation steps of step 3 are as follows:
[0053] The Backbone in YOLOX was changed to Swin Transformer, and the overall network architecture adopted a hierarchical construction method. By downsampling the feature maps, feature sizes after 8x, 16x and 32x downsampling were extracted.
[0054] During feature extraction, a sliding window is used to calculate the attention between pixels, which reduces the amount of computation while enabling information exchange between multiple windows.
[0055] The specific implementation steps of step 4 are as follows:
[0056] The CSPnet structure is introduced into spatial pyramid pooling. This module has a residual edge-assisted optimization and feature extraction to increase the depth of the model.
[0057] By adopting the concept of SPPF and optimizing the pooling structure of SPP, the computation speed is improved without changing the number of parameters.
[0058] The specific implementation steps of step 5 are as follows:
[0059] The Coordinate Attention (CA) mechanism is introduced to optimize the network and improve its accuracy. CA consists of two parts: coordinate information embedding and coordinate attention generation. The CA module divides the input feature map into two directions, width and height, and performs global average pooling on each direction to obtain feature outputs in two directions. The output formula is as follows:
[0060] (1)
[0061] (2)
[0062] Where c is the channel, h is the height, w is the width, and X is the input feature map; the feature maps in the two directions are concatenated, and then transformed into a 1*1 shared convolution to obtain a new feature output f, the formula for f is as follows:
[0063] (3)
[0064] in, For activation function, The feature map after normalization; after obtaining f, it needs to be sliced into two separate tensors along the spatial dimension. and Using two 1x1 convolutions and The two tensors are transformed to have the same number of channels as the input feature map X, and finally, the attention weights for the width and height of the feature map are obtained by passing them through the sigmoid activation function. The attention weight formula is as follows:
[0065] (4)
[0066] (5)
[0067] After obtaining the attention weights in the width and height directions, a weighted calculation is performed on the original feature map to finally obtain a feature map with attention weights in the width and height directions. The final output formula is as follows:
[0068] (6)
[0069] The feature map provides attention weights in the width and height directions, which improves the model's accuracy.
[0070] The specific implementation steps of step 6 are as follows:
[0071] A traffic flow model is constructed using a Poisson process to describe the passenger arrival process. The probability of the number of people waiting for the elevator in the interval [t1, t1+tτ] is as follows:
[0072] (7)
[0073] Where k is The number of passengers detected in the elevator lobby by the target detection model within a time period. for The passenger arrival rate within a time period is expressed as follows:
[0074] (8)
[0075] The passenger distribution data extracted by the target detection module is input into the traffic flow model to predict passengers' target floors and calculate passengers' waiting time, elevator travel time, and the time the elevator stays on a certain floor under different passenger volumes. The system records these time parameters and inputs them into the data management module. The data management module performs traffic pattern recognition based on the parameters and selects the appropriate traffic mode according to the current proportion of passengers. When there are more passengers on lower floors and the upward elevator call signal is responded to first, the system will select the upward peak traffic mode; when there are more passengers on higher floors and the downward elevator call signal is responded to first, the system will select the downward peak traffic mode; when passengers are relatively dispersed across different floors, the system will select the inter-floor traffic mode; and when there are fewer passengers, the system will select the idle traffic mode.
Claims
1. A high-precision elevator traffic pattern recognition method for target detection, characterized in that, The method constructs a network model for passenger traffic pattern recognition. The traffic pattern recognition module obtains real-time passenger flow information based on object detection algorithms and traffic flow model predictions, and selects the current traffic mode based on the passenger flow information. Specifically, it includes the following steps: 1) Create the passenger dataset required for the elevator system and complete the dataset annotation; 2) Configure the network environment using the deep learning PyTorch framework, and complete the construction of the YOLOX model within this environment; 3) Improve YOLOX's backbone by using the self-attention mechanism Swing Transformer to enhance feature extraction; 4) Improve the spatial pyramid pooling module of YOLOX by adopting the CSPnet structure and SPPF concept to enhance the depth and computational speed of the spatial pyramid pooling module; 5) Improve YOLOX's Neck by using a Coordinate Attention mechanism to enhance feature fusion at different sizes; 6) Construct a traffic flow model for the system and select the current traffic mode based on the passenger distribution obtained by the target detection module; The passenger dataset required for building the elevator system is used to process the image dataset input to the network to complete the construction of the object detection model. The model architecture consists of four parts: Input, Backbone, Neck, and Head. The Input layer preprocesses the input images, using Mosaic and Mixup data augmentation methods to improve the diversity of the dataset and enhance the robustness of the model. The Backbone uses the CSPDarknet model, which adds a CSPnet structure to Darknet53. This structure splits the input feature map into two parts: the backbone is divided into the original residual structure and subjected to a series of convolution operations, while the branch parts are not processed. Finally, the two branches are concatenated, and the fused features are stacked. The Neck uses the PAnet path aggregation network structure, which consists of a bottom-up upsampling tower and a top-down downsampling tower. It upsamples and downsamples features of different sizes, concatenates them when they are the same size, and fuses them through convolution to extract feature information of different sizes. Decoupled features are used. The head uses three sizes of features from the feature fusion network for prediction, and improves model accuracy by using Anchor-free and SimOTA-based methods. The improvement to spatial pyramid pooling aims to enhance passenger information across multiple dimensions during the feature fusion stage. It employs a CA attention mechanism to strengthen the features fused at different sizes. This CA attention mechanism is added to the improved YOLOX-Swin Transformer network. This mechanism embeds positional information into channel attention, obtaining attention weights on the height and width of the feature map. Finally, by incorporating these attention weights into the original feature map, the network's detection accuracy is improved. The CA attention mechanism formula is as follows: （1）。 2. The high-precision elevator traffic pattern recognition method for target detection according to claim 1, characterized in that, The target detection model is trained using a dataset of passenger images from real elevator environments, retaining the optimal weights during training. Video recordings from monitoring equipment are collected and processed frame by frame to obtain passenger images from various elevator environments. First, these images are converted to JPG format based on dataset annotations. Then, the labelimg annotation tool is used to annotate passenger categories in all image datasets, and the labels are output in XML format. When inputting into the network, the XML file containing the annotation information is input into the target detection model in TXT format. During network training, the data is divided into training and test sets. 90% of the dataset is used as the training set, and the remaining 10% is used as the test set.
3. The high-precision elevator traffic pattern recognition method for target detection according to claim 1, characterized in that, The traffic flow model uses the Backbone of the Swin Transformer optimized model. By downsampling the feature map, feature maps with feature sizes of 8x, 16x, and 32x are extracted. Multi-window information exchange is carried out using a sliding window to improve the model's attention to passengers.
4. The high-precision elevator traffic pattern recognition method for target detection according to claim 3, characterized in that, The optimized model's backbone is an improvement upon the backbone. To further enhance feature extraction, spatial pyramid pooling is improved, and CSPnet is used to expand the model's receptive field. A CSP structure is introduced into SPP, which has a residual edge to assist in optimization and feature extraction, thereby increasing the model's depth. Based on the idea of SPPF, the pooling structure of SPP is optimized, improving the computation speed without changing the number of parameters.
5. The high-precision elevator traffic pattern recognition method for target detection according to claim 3, characterized in that, The Backbone of the optimized model performs various optimizations on the model, trains and tests the performance of the object detection model, and realizes traffic pattern recognition; sets some parameters such as the number of data sheets input into the network at one time, the number of training rounds, and worker threads, and begins training the model; After the model training is complete, check whether the model's performance metrics are reasonable, turn on the camera to detect the input data, and check the confidence of the predicted bounding boxes; The elevator system uses a target detection network to extract passenger data. The system records the number of people waiting on each floor and the number of people riding in the elevator car. At intervals, it records the average riding time, average waiting time, and dwell time on the target floor. The data is then transmitted to the information management module for elevator scheduling optimization. The traffic pattern recognition module obtains the pedestrian flow data from the target detection and selects the appropriate traffic mode based on the real-time pedestrian flow data.