Escalator passenger flow detection method and system

Abstract

The application discloses an escalator passenger flow detection method and system, and the method comprises the following steps: step one, real-time video monitoring data on the escalator or at the front end of the escalator is acquired, and the acquired real-time video monitoring data is converted into image data; step two, a multi-scale adaptive passenger flow detection model MACD is used to identify and detect the people in the image frame, and time series data is extracted by calculating the number of pixels identified as people by the MACD model in each time step; and step three, a multi-scale attention Transformer model is used to detect the passenger flow of the time series data. The escalator passenger flow is detected in real time, so as to provide data support for predicting the passenger flow of the escalator at different time periods, and the purpose of predicting the escalator passenger flow in advance is achieved, so that the operation state of the escalator can be adjusted in a timely manner, the carrying capacity of the escalator is improved, and the failure probability of the escalator is reduced.

Description

Technical Field

[0001] This invention relates to the field of escalator passenger flow detection technology, and in particular to an escalator passenger flow detection method and system. Background Technology

[0002] Escalators and moving walkways are widely used in shopping malls, subways, airports, supermarkets, and other places, providing great convenience for passengers and serving as an important mode of transportation. However, excessive passenger numbers on escalators and moving walkways can easily lead to speeding, reverse rotation, and in severe cases, even breakage of the drive chain or drive shaft, endangering passenger and property safety. Monitoring the number of passengers on the steps and treads allows for accurate understanding of passenger flow, enabling timely measures such as speed regulation (e.g., dual-speed escalators in subways operate at 0.65 m / s during high passenger flow and 0.5 m / s during low passenger flow, with standby operation during periods of low usage) and flow control, which can alleviate traffic congestion and save energy.

[0003] When an escalator malfunctions, it needs to be stopped, the fault diagnosed, and then restarted. This cycle of malfunction occurrence → stoppage → troubleshooting → normal operation consumes a significant amount of time. A major cause of escalator malfunctions is the mismatch between the escalator's operating speed and passenger flow. For example, operating the escalator at a low speed during peak passenger flow extends its high-load operating time, while operating it at a high speed during low passenger flow accelerates hardware aging and leads to malfunctions. Therefore, matching the escalator's operating speed to passenger flow in real time can effectively extend its lifespan and reduce the failure rate. Existing dual-speed escalators, with only two speed levels (excluding standby operation), struggle to meet these requirements.

[0004] However, there is currently no means to detect the real-time passenger flow of escalators, and the escalator operating speed based on passenger flow conditions by maintenance personnel does not meet the actual usage requirements of the scenario. Summary of the Invention

[0005] This invention provides a method and system for detecting passenger flow on escalators. Its advantage is that it can detect passenger flow on escalators in real time, thereby providing data support for adjusting the operating speed of escalators in real time according to passenger flow, and also providing data support for predicting passenger flow on escalators at different times. This is conducive to achieving the goal of predicting passenger flow on escalators in advance, thereby adjusting the operating status of escalators in a timely manner, improving the carrying capacity of escalators and reducing the probability of escalator failure.

[0006] The technical solution of the present invention is as follows:

[0007] This invention provides a method for detecting passenger flow on escalators, comprising:

[0008] Step 1: Acquire real-time video surveillance data on or at the front of the escalator, and convert the acquired real-time video surveillance data into image data;

[0009] Step 2: Use the multi-scale adaptive crowd detection model (MACD) to identify and detect people in the image frames. Extract time series data by calculating the number of pixels identified as people by the MACD model in each time step.

[0010] Step 3: Use the multi-scale attention Transformer model to detect pedestrian traffic on the time series data.

[0011] Furthermore, in step one, the obtained image data undergoes preprocessing:

[0012] Scaling the image;

[0013] The pixel values ​​of the scaled image are normalized.

[0014] Furthermore, in step two, the multi-scale adaptive pedestrian detection model (MACD) includes:

[0015] The feature extraction module is used to extract a set of feature maps of different depths from the input image;

[0016] The multi-scale feature fusion module is used to fuse feature maps obtained at different depths;

[0017] It also includes an attention-guided feature fusion module, which introduces an attention mechanism to guide feature fusion.

[0018] Furthermore, the feature extraction module is a convolutional neural network (CNN), which includes multiple convolutional layers, activation layers, and pooling layers. Each convolutional layer uses a 3x3 convolutional kernel and is paired with padding with a stride of 1 to maintain the size of the feature map. ReLU is used as the activation function, and a 2x2 max pooling layer is used after each convolutional layer to reduce the size of the feature map and obtain a set of feature maps of different depths.

[0019] The input image is I, the convolution operation is C, the activation function is ReLU, and the max pooling is P. For each layer, perform the following operations:

[0020] Fi = P(R(C(I, Ki, si, pi)))

[0021] Where Fi is the feature map of the i-th layer, Ki is the convolution kernel of the i-th layer, si is the stride, and pi is the padding size;

[0022] Through the above operations, a set of feature maps {Fi} with different depths are obtained.

[0023] Furthermore, in the multi-scale feature fusion module, a feature pyramid network (FPN) is used to fuse the feature maps obtained from the feature extraction module.

[0024] In the Feature Pyramid Network (FPN), the feature maps with greater depth are first reduced in dimensionality through 1x1 convolutional layers, and then fused with the feature maps with less depth through upsampling and element-wise addition.

[0025] Select three feature maps F of different depths from a convolutional neural network (CNN). l , F m , F s These correspond to large, medium, and small scales, respectively; feature fusion can be expressed by the following formula:

[0026] F' l = C(F l , K 1x1 )

[0027] F' m = F' l ⊕Up(F m )

[0028] F' s = F' m ⊕Up(F s )

[0029] Where F' l , F' m , F' s For the fused feature map, K 1x1 The kernel is 1x1, ⨁ represents element-wise addition, and Up represents upsampling.

[0030] Furthermore, in the attention-guided feature fusion module, a 1x1 convolutional layer and a softmax layer are first used to compute the attention weights for each feature map:

[0031] A i = Softmax(C(F' i , K 1x1 ));

[0032] Then, the attention weights are used to weight and fuse the different feature maps to obtain the final fused feature map:

[0033] F'' s = A l * F' l + A m * F' m + A s * F' s

[0034] Where A i Let F'' be the attention weight for the i-th feature map, and Softmax be the softmax function. s This is the final fused feature map.

[0035] Furthermore, in step two, the final fused feature map is processed using an adaptive thresholding algorithm to output the final person detection result, including:

[0036] First, a histogram is calculated for the response intensities of all fused feature maps. Then, the Otsu method is used to find an optimal threshold, and regions with response intensities greater than this threshold are identified as people.

[0037] In the feature fusion map F'' s In the process, each pixel position p is scored to obtain a score map S, where S(p) = F'' s (p);

[0038] Calculate the histogram H of the rating chart:

[0039] H = hist(S)

[0040] Here, hist is the histogram calculation function, which divides the input rating map S into several small intervals and calculates the number of pixels in each small interval;

[0041] Use the Otsu method to find an optimal threshold T:

[0042] T = Otsu(H)

[0043] The Otsu method is an automatic threshold determination method. Its goal is to find a threshold T such that after dividing the histogram into two parts, the variance within each part is minimized. The steps are as follows:

[0044] First, calculate the cumulative sum W and cumulative mean μ of the histogram:

[0045]

[0046]

[0047] Then calculate the inter-class variance σ:

[0048] σ(i) = [μ(T) * W(i) - μ(i)] 2 / [W(i) * (1 - W(i))]

[0049] Finally, find the i that maximizes σ(i) as the optimal threshold T:

[0050] ;

[0051] In the scoring image S, identify all continuous regions greater than the threshold T and generate a bounding box for each region; specifically, first use an image segmentation algorithm to identify all continuous regions in the binary image where S > T:

[0052] Regions = segment(S > T);

[0053] For each region R ∈ Regions, compute its smallest bounding rectangle as the bounding box:

[0054] BBox(R) = minEnclosingRectangle(R);

[0055] The final character detection result is the set of bounding boxes for all regions:

[0056] Detection = {BBox(R) | R ∈ Regions}.

[0057] Furthermore, in step two, the time series data of passenger flow is extracted:

[0058] For each time step t, calculate the number of pixels N(t) identified as passengers by the MACD model:

[0059]

[0060] in, It is the score of pixel p in the score image at time step t, where T is the threshold calculated by the Otsu method;

[0061] Then, the number of these pixels N(t) is used as time series data.

[0062] Furthermore, in step three, a multi-scale attention Transformer model is used to detect pedestrian traffic on the time-series data, including:

[0063] Input embedding: First, the input passenger flow time series data is converted into embedding vectors through an embedding layer.

[0064] ;

[0065] X(t) = Embedding(N(t));

[0066] Position encoding: Then the position encoding is added to the embedding vector to obtain The goal of positional encoding is to enable the model to understand the relative position between time steps. Positional encoding is calculated using the following formula:

[0067]

[0068]

[0069] Where t is the time step, i is the dimension index of the embedding vector, and d model This is the dimension of the embedding vector; then the positional encoding PE(t) is added to the embedding vector X(t):

[0070] X'(t) = X(t) + PE(t);

[0071] Multi-scale self-attention: For each time step t, calculate the attention weights at different scales. At each scale, calculate the attention weight A(t, s).

[0072] A(t, s) = softmax((X'(t) * X') / sqrt(s))

[0073] Then, the attention weights at different scales are summed to obtain the final attention weight A(t):

[0074] A(t) = sum s (A(t, s))

[0075] Finally, the input vector is weighted using this attention weight to obtain a new representation X''(t) for each time step:

[0076] X''(t) = sum i (A(t, i) * X'(i));

[0077] Feedforward Neural Network: The output of multi-scale self-attention is input into a feedforward neural network to obtain the final representation Y(t) at each time step.

[0078] Y(t) = FFN(X''(t))

[0079] Output linear layer: Finally, a linear layer is used to convert the final representation into the people flow detection Y'(t):

[0080] Y'(t) = Linear(Y(t)).

[0081] This invention provides an escalator passenger flow detection system, comprising:

[0082] The data acquisition module is used to acquire real-time video monitoring data on or at the front end of the escalator, including one or more video acquisition devices;

[0083] The edge computing module includes a data preprocessing module, a passenger detection module, and a passenger flow detection module. The data preprocessing module segments the acquired video data into frames to obtain images, scales the images, and normalizes the pixel values ​​of the scaled images. The passenger detection module runs a multi-scale adaptive crowd detection model (MACD) to detect passengers in the image frames. The passenger flow detection module runs a multi-scale attention Transformer model to acquire passenger flow data.

[0084] The system also includes a control module, which is deployed in the cloud or locally. The control module receives data from the edge computing module and communicates with the escalator's control system. The control module adjusts the escalator's operating status based on the calculation results of the edge computing module.

[0085] In summary, the beneficial effects of this application are as follows:

[0086] 1. This patented method employs a multi-scale adaptive crowd detection model (MACD) and a passenger flow prediction model based on a multi-scale attention Transformer. Both models are designed to handle dense and complex crowd scenarios. The MACD model, through multi-scale feature fusion and attention-guided feature fusion, can more accurately detect people in images, maintaining high accuracy even in dense crowds or occlusion conditions. The passenger flow prediction model based on a multi-scale attention Transformer can capture patterns at different scales in passenger flow data, thus more accurately predicting future passenger flow. This high-precision passenger flow detection and prediction can help escalators and moving walkways more accurately predict and adapt to upcoming passenger flow changes, thereby improving operational efficiency, reducing energy consumption, and enhancing the user experience.

[0087] 2. This patent performs data processing and analysis on edge computing devices, which reduces data transmission latency and decreases the demand for network bandwidth. This design is ideal for passenger flow detection and prediction in limited network environments, such as shopping malls and subway stations. In these locations, network bandwidth may be consumed by a large number of user devices, and data transmission latency can affect the response speed of escalators and moving walkways. By performing data processing and analysis on edge computing devices, the method of this patent can achieve rapid passenger flow detection and prediction, thereby improving the response speed of escalators and moving walkways and enhancing the user experience. Attached Figure Description

[0088] Figure 1This is a schematic diagram illustrating the principle of the escalator passenger flow detection method in this invention;

[0089] Figure 2 This is a flowchart illustrating the escalator passenger flow detection method of the present invention;

[0090] Figure 3 This is a schematic diagram of video data acquisition in this invention;

[0091] Figure 4 This is a flowchart illustrating the multi-scale adaptive crowd detection model in this invention.

[0092] Figure 5 This is a flowchart illustrating the multi-scale attention Transformer model in this invention.

[0093] Figure 6 This is a schematic diagram of the computation process of the multi-scale attention Transformer model in this invention;

[0094] Figure 7 This is a schematic diagram of the escalator passenger flow detection system in this invention. Detailed Implementation

[0095] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0096] In this embodiment of the invention, an escalator is used as an example for specific description. It should be understood that moving walkways are also applicable to this invention.

[0097] In one embodiment of the present invention, reference is made to Figures 1-2 A method for detecting passenger flow on escalators is provided, comprising:

[0098] Step 1: Acquire real-time video surveillance data on or at the front of the escalator, and convert the acquired real-time video surveillance data into image data;

[0099] Step 2: Use the multi-scale adaptive crowd detection model (MACD) to identify and detect people in the image frames. Extract time series data by calculating the number of pixels identified as people by the MACD model in each time step.

[0100] Step 3: Use the multi-scale attention Transformer model to detect pedestrian traffic on the time series data.

[0101] like Figure 3As shown, in step one, the real-time video monitoring data is obtained by a monitoring device, which is a camera 11 installed above the escalator. The camera 11 is fixedly suspended from the floor slab 3. It is understood that the camera can also be installed at the entrance of the escalator. In the escalator 1, the steps 5 and the handrail 6 move synchronously, moving passengers and thus being captured by the camera 11.

[0102] In step one, the obtained video data is converted into image data. Specifically, the obtained video data is divided into frames to form multiple still image data. Assuming the video plays at a speed of f frames per second, then after T seconds, N = f*T images will be obtained.

[0103] The acquired image data undergoes preprocessing, which includes scaling the images and normalizing the pixel values ​​of the scaled images. For each frame of the image, it needs to be converted to a format acceptable to the deep learning model. For example, the image size is adjusted to (M, M, 3), where M is the height and width of the image, and 3 is the number of RGB color channels. Then, the pixel values ​​of the image need to be normalized to [-1, 1] or [0, 1]. Assuming that the original pixel values ​​of image I are in [0, 255], it can be normalized to [-1, 1] or [0, 1]. normalized = I / 127.5 - 1 or I normalized = I / 255 to normalize pixel values.

[0104] In step two, the multi-scale adaptive crowd detection model (MACD) includes:

[0105] The feature extraction module is used to extract a set of feature maps of different depths from the input image;

[0106] The multi-scale feature fusion module is used to fuse feature maps obtained at different depths;

[0107] It also includes an attention-guided feature fusion module, which introduces an attention mechanism to guide feature fusion.

[0108] The feature extraction module is a convolutional neural network (CNN), which includes multiple convolutional layers, activation layers, and pooling layers. Each convolutional layer uses a 3x3 convolutional kernel and padding with a stride of 1 to maintain the size of the feature map. ReLU is used as the activation function, and a 2x2 max pooling layer is used after each convolutional layer to reduce the size of the feature map and obtain a set of feature maps of different depths.

[0109] The input image is I, the convolution operation is C, the activation function is ReLU, and the max pooling is P. For each layer, perform the following operations:

[0110] Fi = P(R(C(I, Ki, si, pi)))

[0111] Where Fi is the feature map of the i-th layer, Ki is the convolution kernel of the i-th layer, si is the stride, and pi is the padding size. Through the above operations, a set of feature maps {Fi} of different depths is obtained.

[0112] These feature maps contain multi-scale information from the input image, which can be used for subsequent feature fusion and person detection. Specifically, shallow feature maps mainly contain local and detailed information (such as edges and textures), while deep feature maps mainly contain global and abstract information (such as the overall shape of objects). This information is crucial for accurately detecting people and handling complex scenes (such as dense and occluded scenes).

[0113] In the multi-scale feature fusion module, detection is performed on feature maps of different depths, and these detection results are fused together. Specifically, this invention selects three feature maps of different depths from the CNN, corresponding to large, medium, and small scales, respectively. A Feature Pyramid Network (FPN) is used to fuse the feature maps obtained from the feature extraction module.

[0114] In the Feature Pyramid Network (FPN), the feature maps with greater depth are first reduced in dimensionality through 1x1 convolutional layers, and then fused with the feature maps with less depth through upsampling and element-wise addition.

[0115] Select three feature maps F of different depths from a convolutional neural network (CNN). l , F m , F s These correspond to large, medium, and small scales, respectively; feature fusion can be expressed by the following formula:

[0116] F' l = C(F l , K 1x1 )

[0117] F' m = F' l ⊕Up(F m )

[0118] F' s = F' m ⊕Up(F s )

[0119] Where F' l , F' m , F' s For the fused feature map, K 1x1 The kernel is 1x1, ⨁ represents element-wise addition, and Up represents upsampling.

[0120] In the Attention-Guided Feature Fusion (AFF) module, an attention mechanism will be introduced to guide feature fusion. First, a 1x1 convolutional layer and a softmax layer are used to compute the attention weights for each feature map:

[0121] A i = Softmax(C(F' i , K 1x1 ));

[0122] Then, the attention weights are used to weight and fuse the different feature maps to obtain the final fused feature map:

[0123] F'' s = A l * F' l + A m * F' m + A s * F' s

[0124] Where A i Let F'' be the attention weight for the i-th feature map, and Softmax be the softmax function. s This is the final fused feature map.

[0125] In step two, the final fused feature map is processed using an adaptive thresholding algorithm to output the final person detection result, including:

[0126] ① First, calculate the histogram of the response intensities of all fused feature maps. Then, use the Otsu method to find an optimal threshold, and identify regions with response intensities greater than this threshold as people.

[0127] In the feature fusion map F'' s In the process, each pixel position p is scored to obtain a score map S, where S(p) = F'' s (p);

[0128] ② Calculate the histogram H of the rating chart:

[0129] H = hist(S)

[0130] Here, hist is the histogram calculation function, which divides the input rating map S into several small intervals and calculates the number of pixels in each small interval;

[0131] ③ Use the Otsu method to find an optimal threshold T:

[0132] T = Otsu(H)

[0133] The Otsu method is an automatic threshold determination method. Its goal is to find a threshold T such that after dividing the histogram into two parts, the variance within each part is minimized. The steps are as follows:

[0134] First, calculate the cumulative sum W and cumulative mean μ of the histogram:

[0135]

[0136]

[0137] Then calculate the inter-class variance σ:

[0138] σ(i) = [μ(T) * W(i) - μ(i)] 2 / [W(i) * (1 - W(i))]

[0139] Finally, find the i that maximizes σ(i) as the optimal threshold T:

[0140] ;

[0141] ④ In the scoring image S, find all continuous regions greater than the threshold T, and generate a bounding box for each region; specifically, first use an image segmentation algorithm to find all continuous regions in the binary image where S > T:

[0142] Regions = segment(S > T);

[0143] ⑤ For each region R ∈ Regions, calculate its smallest bounding rectangle as the bounding box:

[0144] BBox(R) = minEnclosingRectangle(R);

[0145] ⑥ The final character detection result is the set of bounding boxes for all regions:

[0146] Detection = {BBox(R) | R ∈ Regions}.

[0147] In summary, the computational process of the Multi-scale Adaptive Crowd Detection (MACD) model is as follows: Figure 4 As shown.

[0148] Step two involves extracting the time series data of passenger flow, including:

[0149] For each time step t, calculate the number of pixels N(t) identified as passengers by the MACD model:

[0150]

[0151] in, It is the score of pixel p in the score image at time step t, where T is the threshold calculated by the Otsu method;

[0152] Then, the number of these pixels N(t) is used as time series data.

[0153] In step three, a multi-scale attention Transformer model is used to detect pedestrian flow in the time series data. While traditional Transformer models demonstrate excellent performance in time series forecasting, there is still significant room for improvement in pedestrian flow prediction. This is because pedestrian flow data typically contains patterns at various scales. For example, pedestrian flow may be influenced by multiple factors such as hourly work schedules and daily weather changes. Traditional Transformer models cannot simultaneously capture these patterns at different scales. Therefore, this invention proposes a multi-scale attention-based Transformer model for pedestrian flow prediction, comprising:

[0154] Input embedding: First, the input passenger flow time series data is converted into embedding vectors through an embedding layer.

[0155]

[0156] X(t) = Embedding(N(t));

[0157] Position encoding: Then the position encoding is added to the embedding vector to obtain The goal of positional encoding is to enable the model to understand the relative position between time steps. Positional encoding is calculated using the following formula:

[0158]

[0159]

[0160] Where t is the time step, i is the dimension index of the embedding vector, and d model This is the dimension of the embedding vector; then the positional encoding PE(t) is added to the embedding vector X(t):

[0161] X'(t) = X(t) + PE(t);

[0162] Multi-scale self-attention: For each time step t, calculate the attention weights at different scales. At each scale, calculate the attention weight A(t, s).

[0163] A(t, s) = softmax((X'(t) * X') / sqrt(s))

[0164] Then, the attention weights at different scales are summed to obtain the final attention weight A(t):

[0165] A(t) = sum s (A(t, s))

[0166] Finally, the input vector is weighted using this attention weight to obtain a new representation X''(t) for each time step:

[0167] X''(t) = sum i (A(t, i) * X'(i));

[0168] Feedforward Neural Network: The output of multi-scale self-attention is input into a feedforward neural network to obtain the final representation Y(t) at each time step.

[0169] Y(t) = FFN(X''(t))

[0170] Output linear layer: Finally, a linear layer is used to convert the final representation into the people flow detection Y'(t):

[0171] Y'(t) = Linear(Y(t)).

[0172] The calculation process for pedestrian flow prediction based on time series forecasting is as follows: Figure 5 As shown.

[0173] The computational flowchart of the multi-scale attention Transformer model is as follows: Figure 6 As shown, historical passenger flow data is input into the multi-scale attention Transformer model. The functions of each module of the multi-scale attention Transformer model are as follows:

[0174] 1. Input Embedding: The input module converts historical passenger flow data into embedding vectors.

[0175] 2. Position Encoding: In order to provide the model with positional information in the sequence, each embedding vector will be given a position encoding.

[0176] 3. Multi-scale self-attention: For the input at each time step, this module will calculate attention weights at different scales, and then apply these weights to the corresponding input vector to obtain a weighted input vector.

[0177] 4. Feedforward Depth Layer: This module processes the output of multi-scale self-attention and generates a new vector sequence.

[0178] 5. Encoder: The encoder consists of multiple multi-scale self-attention modules and point-based feedforward network modules, which encode the input.

[0179] 6. Decoder: The decoder consists of multiple multi-scale self-attention modules, encoder-decoder attention modules, and point feedforward network modules. It generates new prediction vectors based on the output of the encoder and the previous output of the decoder.

[0180] 7. Linear Layer and Normalized Exponential Function Layer: The decoder output passes through this module to generate predicted future passenger flow time series data.

[0181] The procedure flow of the multi-scale attention Transformer model is as follows: First, encoder input embedding is performed, converting the input pedestrian flow time series data into an embedding vector through an embedding layer. Second, location information embedding is performed, adding location information to the embedding vector so that the model understands the relative positions between time steps. Next, a multi-scale self-attention mechanism is introduced, calculating the attention weights for each time step at different scales, and obtaining the final weights by superposition. These attention weights are used to weight the input vector, iterating to obtain new time steps. This improved mechanism enables the Transformer model to capture patterns at different scales in pedestrian flow data, thereby more accurately predicting future pedestrian flow. Then, the output of the multi-scale self-attention is subjected to residual connections and layer normalization, and input into a feedforward neural network to obtain the final representation of each time step. Finally, a linear layer and a normalization function are used to convert the final representation of each time step into a pedestrian flow prediction value.

[0182] One embodiment of the present invention provides an escalator passenger flow detection system, with reference to... Figure 7 ,include:

[0183] The data acquisition module is used to acquire real-time video monitoring data on or at the front end of the escalator, including one or more video acquisition devices;

[0184] The edge computing module includes a data preprocessing module, a passenger detection module, and a passenger flow detection module. The data preprocessing module segments the acquired video data into frames to obtain images, scales the images, and normalizes the pixel values ​​of the scaled images. The passenger detection module runs a multi-scale adaptive crowd detection model (MACD) to detect passengers in the image frames. The passenger flow detection module runs a multi-scale attention Transformer model to acquire passenger flow data.

[0185] The escalator also includes a control module, deployed in the cloud or locally, which receives data from the edge computing module and communicates with the escalator's control system. The control module adjusts the escalator's operating status based on the calculation results of the edge computing module. The above description is merely a preferred embodiment of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the inventive concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention.

Claims

1. A method for detecting passenger flow on escalators, characterized in that, include: Step 1: Acquire real-time video surveillance data on or at the front of the escalator, and convert the acquired real-time video surveillance data into image data; Step 2: Use the multi-scale adaptive crowd detection model (MACD) to identify and detect people in the image frames. Extract time series data by calculating the number of pixels identified as people by the MACD model in each time step. The multi-scale adaptive crowd detection model (MACD) includes: The feature extraction module is used to extract a set of feature maps of different depths from the input image; The multi-scale feature fusion module is used to fuse feature maps obtained at different depths; And an attention-guided feature fusion module, which introduces an attention mechanism to guide feature fusion; The feature extraction module is a convolutional neural network (CNN); In the multi-scale feature fusion module, the Feature Pyramid Network (FPN) is used to fuse the feature maps obtained from the feature extraction module. In the attention-guided feature fusion module, the attention weight of each feature map is calculated, and the attention weights are used to weight and fuse different feature maps to obtain the final fused feature map. Step 3: Utilize a multi-scale attention Transformer model to detect pedestrian flow in the time series data. First, convert the input pedestrian flow time series data into an embedding vector through an embedding layer. Then, add the location encoding to the embedding vector. For each time step, calculate the attention weight at different scales, sum the attention weights at different scales to obtain the final attention weight, and use this attention weight to weight the input vector. Input the output of the multi-scale self-attention model into a feedforward neural network to obtain the final representation at each time step. Finally, use a linear layer to convert the final representation into pedestrian flow detection.

2. The method for detecting passenger flow on escalators according to claim 1, characterized in that, In step one, the obtained image data is preprocessed: Scaling the image; The pixel values ​​of the scaled image are normalized.

3. The method for detecting passenger flow on escalators according to claim 1, characterized in that, The feature extraction module is a convolutional neural network (CNN), which includes multiple convolutional layers, activation layers, and pooling layers. Each convolutional layer uses a 3x3 convolutional kernel and padding with a stride of 1 to maintain the size of the feature map. ReLU is used as the activation function, and a 2x2 max pooling layer is used after each convolutional layer to reduce the size of the feature map and obtain a set of feature maps of different depths. The input image is I, the convolution operation is C, the activation function is ReLU, and the max pooling is P. For each layer, perform the following operations: Fi = P(R(C(I, Ki, si, pi))) Where Fi is the feature map of the i-th layer, Ki is the convolution kernel of the i-th layer, si is the stride, and pi is the padding size; Through the above operations, a set of feature maps {Fi} with different depths are obtained.

4. The method for detecting passenger flow on escalators according to claim 3, characterized in that, In the multi-scale feature fusion module, the Feature Pyramid Network (FPN) is used to fuse the feature maps obtained from the feature extraction module. In the Feature Pyramid Network (FPN), the feature maps with greater depth are first reduced in dimensionality through 1x1 convolutional layers, and then fused with the feature maps with less depth through upsampling and element-wise addition. Select three feature maps F of different depths from a convolutional neural network (CNN). l , F m , F s These correspond to large, medium, and small scales, respectively; feature fusion is expressed by the following formula: F' l = C(F l , K 1x1 ) F' m = F' l ⊕Up(F m ) F' s = F' m ⊕Up(F s ) Where F' l , F' m , F' s For the fused feature map, K 1x1 The kernel is 1x1, ⨁ represents element-wise addition, and Up represents upsampling.

5. The method for detecting passenger flow on escalators according to claim 4, characterized in that, In the attention-guided feature fusion module, a 1x1 convolutional layer and a softmax layer are first used to compute the attention weights for each feature map: A i = Softmax(C(F' i , K 1x1 )); Then, the attention weights are used to weight and fuse the different feature maps to obtain the final fused feature map: F'' s = A l * F' l + A m * F' m + A s * F' s Where A i Let F'' be the attention weight for the i-th feature map, and Softmax be the softmax function. s This is the final fused feature map.

6. The method for detecting passenger flow on escalators according to claim 5, characterized in that, In step two, the final fused feature map is processed using an adaptive thresholding algorithm to output the final person detection result, including: First, a histogram is calculated for the response intensities of all fused feature maps. Then, the Otsu method is used to find an optimal threshold, and regions with response intensities greater than this threshold are identified as people. In the feature fusion map F'' s In the process, each pixel position p is scored to obtain a scoring map S, where S(p) = F'' s (p); Calculate the histogram H of the rating chart: H = hist(S) Here, hist is the histogram calculation function, which divides the input rating map S into several small intervals and calculates the number of pixels in each small interval; Use the Otsu method to find an optimal threshold T: T = Otsu(H) The Otsu method is an automatic threshold determination method. Its goal is to find a threshold T such that after dividing the histogram into two parts, the variance within each part is minimized. The steps are as follows: First, calculate the cumulative sum W and cumulative mean μ of the histogram: ； ； Then calculate the inter-class variance σ: σ(i) = [μ(T) * W(i) - μ(i)] 2 / [W(i) * (1 - W(i))] Finally, find the i that maximizes σ(i) as the optimal threshold T: ； In the scoring image S, identify all continuous regions greater than the threshold T and generate a bounding box for each region; specifically, first use an image segmentation algorithm to identify all continuous regions in the binary image where S > T: Regions = segment(S > T); For each region R ∈ Regions, compute its smallest bounding rectangle as the bounding box: BBox(R) = minEnclosingRectangle(R); The final character detection result is the set of bounding boxes for all regions: Detection = {BBox(R) | R ∈ Regions}.

7. The method for detecting passenger flow on escalators according to claim 6, characterized in that, In step two, extract the time series data of passenger flow: For each time step t, calculate the number of pixels N(t) identified as passengers by the MACD model: ； in, It is the score of pixel p in the score image at time step t, where T is the threshold calculated by the Otsu method; Then, the number of these pixels N(t) is used as time series data.

8. The method for detecting passenger flow on escalators according to claim 7, characterized in that, In step three, a multi-scale attention Transformer model is used to detect pedestrian traffic on the time series data, including: Input embedding: First, the input passenger flow time series data is converted into embedding vectors through an embedding layer. ； X(t) = Embedding(N(t)); Position encoding: Then the position encoding is added to the embedding vector to obtain The goal of positional encoding is to enable the model to understand the relative position between time steps. Positional encoding is calculated using the following formula: ； ； Where t is the time step, i is the dimension index of the embedding vector, and d model This is the dimension of the embedding vector; then the positional encoding PE(t) is added to the embedding vector X(t): X'(t) = X(t) + PE(t); Multi-scale self-attention: For each time step t, calculate the attention weights at different scales. At each scale, calculate the attention weight A(t, s). A(t, s) = softmax((X'(t) * X') / sqrt(s)) Then, the attention weights at different scales are summed to obtain the final attention weight A(t): A(t) = sum s (A(t, s)) Finally, the input vector is weighted using this attention weight to obtain a new representation X''(t) for each time step: X''(t) = sum i (A(t, i) * X'(i)); Feedforward Neural Network: The output of multi-scale self-attention is input into a feedforward neural network to obtain the final representation Y(t) at each time step. Y(t) = FFN(X''(t)) Output linear layer: Finally, a linear layer is used to convert the final representation into the people flow detection Y'(t): Y'(t) = Linear(Y(t)).

9. An escalator passenger flow detection system, characterized in that, To implement the steps in the method as described in any one of claims 1-8, including: The data acquisition module is used to acquire real-time video monitoring data on or at the front end of the escalator, including one or more video acquisition devices; The edge computing module includes a data preprocessing module, a passenger detection module, and a passenger flow detection module. The data preprocessing module segments the acquired video data into frames to obtain images, scales the images, and normalizes the pixel values ​​of the scaled images. The passenger detection module runs a multi-scale adaptive crowd detection model (MACD) to detect passengers in the image frames. The passenger flow detection module runs a multi-scale attention Transformer model to acquire passenger flow data. The system also includes a control module, which is deployed in the cloud or locally. The control module receives data from the edge computing module and communicates with the escalator's control system. The control module adjusts the escalator's operating status based on the calculation results of the edge computing module.

Images