A passenger behavior recognition method for subway platforms with high-density passenger flow scenarios
By optimizing the combination of YOLOv8 network and feature extraction model, the accuracy of passenger behavior recognition in high-density passenger flow scenarios is solved, accurate identification of passenger behavior and early warning of safety events is achieved, and subway operation safety is improved.
Patent Information
- Application Number
- CN202410146328.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-02-01
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2044-02-01
AI Technical Summary
The prior art is difficult to accurately identify the diverse behaviors of subway station passengers in high-density passenger flow scenarios, especially when shading and facilities are blocked, and it is impossible to effectively deal with passengers such as crowding and stranding, affecting the safety of subway operations.
The optimized YOLOv8 network is adopted, combining the hollow convolutional network, Transformer network and contrast learning loss, and the compact global features of passengers are extracted and the occlusions are separated to achieve accurate identification of passenger behavior.
It improves the real-time identification accuracy of platform passenger behavior, reduces the frequency of dangerous events, and enhances the operational safety and reliability of the subway system.
Smart Images

Figure CN118015701B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of video recognition technology, and in particular to a method for identifying passenger behavior on a subway platform in a high-density passenger flow scenario. Background Art
[0002] As the distribution center of passenger flow in the subway system, the subway station platform often needs to carry high-density and highly mobile passenger flow in a small space. The behavior of platform passengers is complex and changeable, and directly affects the subway trains. Since the operation of subway trains needs to strictly follow the train operation diagram, small disturbances in passenger behavior may have a great impact on the operation of trains. The current subway monitoring system lacks the ability to identify the diverse and complex behaviors of individual platform passengers, such as waiting, boarding, getting off, staying, and running, resulting in the inability to effectively deal with the behaviors and states of passengers such as crowding, staying, and wandering at the platform in a timely manner. The currently disclosed patents lack the intelligent identification of platform passenger behavior when high-density passengers block each other during boarding and alighting, or when platform facilities and equipment seriously block passengers, and cannot meet the precise identification of passenger behavior in high-density passenger flow travel scenarios. It can be seen that the intelligent identification of subway platform passenger behavior for high-density passenger flow scenarios plays a very positive role in reducing the incidence of public safety incidents and ensuring the safety of subway operations. In addition, the identification of the real-time status and behavior of platform passengers can provide effective data support for the real-time optimization of train operation and the dynamic optimization of station passenger flow organization plans. Summary of the invention
[0003] The purpose of the present invention is to provide a method for identifying passenger behavior on a subway platform for high-density passenger flow scenarios, which can improve the accuracy of real-time identification of passenger behavior on the platform.
[0004] To achieve the above object, the present invention provides the following solutions:
[0005] A method for identifying passenger behavior on a subway platform in a high-density passenger flow scenario, the method comprising:
[0006] Get the historical surveillance video stream of the subway platform;
[0007] According to the multiple frames of images in the historical video stream image that are continuous in time sequence, the passenger numbers, passenger behaviors and passenger positions of all passengers in each frame of the image are annotated to obtain historical images annotated with passenger behavior category information, and the multiple frames of images in the historical video stream image that are continuous in time sequence and the historical images annotated with passenger behavior category information are used as training sets; the passenger behaviors include waiting, getting on, getting off, staying and running; the passenger positions are the coordinates of the center point of the rectangular annotated box of each passenger and the width and height of the rectangular box;
[0008] The optimized YOLOv8 network is trained using the training set to obtain a passenger behavior recognition model; the optimized YOLOv8 backbone network is obtained by integrating the YOLOv8 backbone network with the passenger feature extraction model;
[0009] Obtain the real-time surveillance video stream of the target subway platform;
[0010] Each frame of the real-time video stream image is input into a passenger behavior recognition model to obtain an image marked with passenger behavior category information.
[0011] Optionally, the feature extraction model includes a dilated convolutional network, a Transformer network, and contrastive learning loss.
[0012] Optionally, the optimized YOLOv8 network includes an optimized backbone network, a neck structure and a head structure connected in sequence; the optimized backbone network is composed of the last layer of the YOLOv8 backbone network Darknet-53 connected to the feature extraction model.
[0013] Optionally, the last layer of the backbone network Darknet-53 of YOLOv8 is connected to the dilated convolutional network; the dilated convolutional network is connected to the Transformer network; the Transformer network is connected to the contrastive learning loss; the contrastive learning loss is connected to the neck structure; the dilated convolutional network is used to extract the compact global features of passengers in the image output by the last layer of the backbone network Darknet-53 of YOLOv8; the Transformer network is used to perform feature decoupling on the compact global features to obtain relevant features of each numbered passenger and irrelevant features of the passengers; the contrastive learning loss is used to use the relevant features of the passengers as positive samples and the irrelevant features of the passengers as negative samples, to perform discriminative constraints on the relevant features of the passengers and the irrelevant features of the passengers, to separate occluders and background noise from the global features of the passengers, and to obtain the features of the passengers.
[0014] Optionally, the identification method further includes:
[0015] The passenger behavior recognition model is optimized by taking each frame of the real-time video stream image as input and the corresponding image labeled with passenger behavior category information as output.
[0016] Optionally, the identification method further includes:
[0017] According to the image marked with the passenger behavior category information, the number of passengers with the same passenger behavior within a first preset time period is counted; the first preset time period takes the current time as the end time;
[0018] The number of passengers with the same passenger behavior within the first preset time period is compared with a preset first threshold to obtain a first comparison result, and a first alarm signal is generated according to the first comparison result.
[0019] Optionally, the identification method further includes:
[0020] According to the image marked with passenger behavior category information, the number of passengers on the platform at the current moment and the number of passengers of each type of behavior are counted;
[0021] A second comparison result is obtained by comparing the number of all passengers at the current moment with a preset second threshold, and a second alarm signal is generated according to the second comparison result.
[0022] Optionally, the identification method further includes:
[0023] According to the image marked with the passenger behavior category information, the number of passengers with the same passenger number within the running interval between trains in a third preset time period is continuously counted; the third preset time period is a morning peak time period, an evening peak time period, an off-peak time period, a designated time period for large-scale events, or a designated time period for major holidays; the third preset time period ends at the current time;
[0024] A third comparison result is obtained by comparing the number of passengers with a preset third threshold, and a third alarm signal is generated according to the third comparison result.
[0025] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:
[0026] The present invention obtains an image marked with position information and passenger behavior of all passengers by dividing the real-time video stream image into frames and inputting it into a passenger behavior recognition model; the passenger behavior recognition model is obtained by training an optimized YOLOv8 backbone network using a training set; further, the optimized YOLOv8 backbone network is obtained by integrating a feature extraction model into the last layer of the YOLOv8 backbone network; and the feature extraction model includes a hole convolution network, a Transformer model and a contrastive learning module box connected in sequence. The present invention uses a dilated convolutional network to extract compact global features such as color, texture, shape and clothing of passengers in the image, and uses Transformer to perform feature decoupling to obtain passenger-related features (such as color, shape and clothing, etc.) and passenger number-irrelevant features (such as station facilities and equipment); passenger-related features are used as positive samples and passenger-irrelevant features are used as negative samples, and contrastive learning is introduced to perform discriminative constraints on passenger-related features and passenger-irrelevant features, separate obstructions such as station facilities and equipment and background noise from the global features of passengers, suppress the interference of occlusion on passenger number matching, and effectively extract the features of obscured passengers in high-density passenger flow scenes, ultimately improving the accuracy of real-time recognition of platform passenger behavior. BRIEF DESCRIPTION OF THE DRAWINGS
[0027] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0028] Figure 1 The flowchart of the present invention is a method for identifying passenger behavior on a subway platform in a high-density passenger flow scenario;
[0029] Figure 2 This is a schematic diagram of the YOLOv8 network structure optimized by the present invention;
[0030] Figure 3 Schematic diagram of the YOLOv8 backbone network structure optimized by the present invention 。 DETAILED DESCRIPTION
[0031] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0032] The purpose of the present invention is to provide a method for identifying passenger behavior on a subway platform for high-density passenger flow scenarios, which can improve the accuracy of real-time identification of passenger behavior on the platform.
[0033] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.
[0034] Embodiment 1
[0035] like Figure 1 As shown, the present invention provides a method for identifying passenger behavior on a subway platform for a high-density passenger flow scenario, the identification method comprising:
[0036] Step S1: Obtain the historical surveillance video stream of the subway platform.
[0037] Step S2: based on the time-sequential multiple-frame images in the historical video stream image, the passenger numbers, passenger behaviors and passenger positions of all passengers in each frame image are annotated to obtain the historical images annotated with the passenger behavior category information, and the time-sequential multiple-frame images in the historical video stream image and the historical images annotated with the passenger behavior category information are used as training sets; the passenger behaviors include waiting, getting on the bus, getting off the bus, staying and running; the passenger positions are the coordinates of the center point of the rectangular annotation box of each passenger and the width and height of the rectangular box.
[0038] Step S3: Apply the training set to train the optimized YOLOv8 network to obtain a passenger behavior recognition model; the optimized YOLOv8 backbone network is obtained by integrating the YOLOv8 backbone network with the passenger feature extraction model.
[0039] Specifically, the feature extraction model includes a dilated convolutional network, a Transformer network and contrastive learning loss.
[0040] Specifically, Figure 2 As shown, the optimized YOLOv8 network includes an optimized backbone network, a neck structure and a head structure connected in sequence; the optimized backbone network is composed of the last layer of the YOLOv8 backbone network Darknet-53 connected to the feature extraction model.
[0041] Furthermore, if Figure 3As shown, the last layer of the backbone network Darknet-53 of YOLOv8 is connected to the dilated convolutional network; the dilated convolutional network is connected to the Transformer network; the Transformer network is connected to the contrastive learning loss; the contrastive learning loss is connected to the neck structure; the dilated convolutional network is used to extract the compact global features of the passengers in the image output by the last layer of the backbone network Darknet-53 of YOLOv8; the Transformer network is used to perform feature decoupling on the compact global features to obtain relevant features of each numbered passenger and irrelevant features of the passengers; the contrastive learning loss is used to use the relevant features of the passengers as positive samples and the irrelevant features of the passengers as negative samples, to perform discriminative constraints on the relevant features of the passengers and the irrelevant features of the passengers, to separate the occluders and background noise from the global features of the passengers, and to obtain the features of the passengers.
[0042] In practical applications, a feature extraction model is applied to extract passenger features in high-density passenger flow scenarios. Specifically, the feature extraction model uses a dilated convolutional network to extract compact global features such as color, texture, shape, and clothing of passengers in the image, and uses Transformer for feature decoupling to obtain passenger-related features (such as color, shape, and clothing) and passenger number-irrelevant features (such as station facilities and equipment). Passenger-related features are used as positive samples, and passenger-irrelevant features are used as negative samples. Contrastive learning is introduced to perform discriminative constraints on passenger-related features and passenger-irrelevant features, separate occluders such as station facilities and equipment and background noise from the global features of passengers, suppress the interference of occlusion on passenger number matching, and effectively extract the features of occluded passengers in high-density passenger flow scenarios.
[0043] The passenger feature extraction module in high-density passenger flow scenarios is integrated into the last layer of the YOLOv8 backbone network Darknet-53 to realize intelligent recognition of passenger behavior in high-density passenger flow scenarios; the image and the corresponding labeled data pairs are brought into the passenger behavior recognition model for model training, and the training is stopped when the loss function value converges on both the training set and the validation set; finally, the real-time video stream is frame-by-frame input into the trained passenger behavior recognition model, and the passenger behavior (waiting, getting on the bus, getting off the bus, staying, running, etc.) of each frame image in the real-time video stream is identified and the behavior recognition model is optimized in a rolling manner.
[0044] In practical applications, each frame of the real-time video stream image is used as input, and the corresponding image labeled with passenger behavior category information is used as output to continuously optimize the passenger behavior recognition model.
[0045] Step S4: Obtain the real-time monitoring video stream of the target subway platform.
[0046] Step S5: input each frame of the real-time video stream image into the passenger behavior recognition model to obtain an image marked with passenger behavior category information.
[0047] In practical applications, we first collect video stream data from surveillance cameras on subway station platforms and divide the video data into image sets. Combining the passenger behavior characteristics and location information in multiple frames of images in a time series, we use manual annotation methods to mark the behavior categories and locations of all passengers in each frame of the image set. Specifically, the behavior categories of passengers on the platform include: waiting, getting on, getting off, staying, and running. The location of each passenger is represented by a rectangular box, which is represented by four coordinate values, namely the coordinates of the center point of the rectangular box and the width and height of the rectangular box.
[0048] As a specific implementation, the identification method further includes:
[0049] According to the image marked with the passenger behavior category information, the number of passengers with the same passenger behavior within a first preset time period is counted; the first preset time period takes the current moment as the end moment.
[0050] The number of passengers with the same passenger behavior in the first preset time period is compared with a preset first threshold to obtain a first comparison result, and a first alarm signal is generated according to the first comparison result. The preset first threshold is a safety baseline for the number of passengers with the same behavior on the platform.
[0051] As a specific implementation, the identification method further includes:
[0052] According to the image marked with the passenger behavior category information, the number of passengers on the platform at the current moment and the number of passengers with each type of behavior are counted.
[0053] The number of all passengers at the current moment is compared with a preset second threshold to obtain a second comparison result, and a second alarm signal is generated according to the second comparison result. The preset second threshold is a safety baseline for the number of passengers on the platform.
[0054] As a specific implementation, the identification method further includes:
[0055] Based on the image marked with passenger behavior category information, the number of passengers with the same passenger number within the running interval between trains in a third preset time period is continuously counted; the third preset time period is the morning peak period, the evening peak period, the off-peak period, the designated time period for large-scale activities or the designated time period for major holidays; the third preset time period ends at the current time.
[0056] The third comparison result is obtained by comparing the number of passengers with a preset third threshold, and a third alarm signal is generated according to the third comparison result. The preset third threshold is the number of passengers stranded on the platform.
[0057] In practical applications, based on the behavior category and location of each passenger at the platform calculated by the behavior recognition model, the number of passengers with each behavior and the total number of passengers on the platform in a continuous period are counted; then, the safety baseline and passenger detention threshold for the number of passengers with each behavior in different operating periods or emergencies (morning peak, evening peak, off-peak period, large-scale activities, major holidays, etc.) are set. When the number of passengers or the number of detentions exceeds the safety threshold, an alarm message containing passenger behavior exceeding the safety threshold is sent to the operation management center. In the present invention, the time period corresponding to different operating periods or emergencies is set as the preset time period; the safety baseline is the first threshold, and the passenger detention threshold is the second threshold.
[0058] The present invention has the following advantages:
[0059] 1. Accurately extract passenger features in high-density passenger flow scenarios: Based on the video stream collected by the subway platform surveillance camera, the present invention proposes a passenger feature extraction module in high-density passenger flow scenarios. Under the premise of ensuring the global compact features of the passengers, it can separate obstructions and background noise from the passenger features, suppress the interference of occlusions on passenger matching, and effectively extract the features of obstructed passengers in high-density passenger flow scenarios.
[0060] 2. Improve the intelligent level of recognition of the diverse behaviors of passengers on subway platforms: Integrate the passenger feature extraction module in high-density passenger flow scenarios into the backbone network of YOLOv8 to realize the intelligent recognition of the diverse behaviors of passengers on subway station platforms, such as waiting, boarding, getting off, staying, and running. This makes up for the lack of intelligent recognition of the diverse behaviors of passengers on the platform when high-density passengers block each other during getting on and off the train or when platform facilities and equipment seriously block passengers.
[0061] 3. The subway platform passenger behavior recognition method for high-density passenger flow scenarios proposed in the present invention can automatically identify platform areas and passenger locations where the number of passengers or the number of detention times exceeds the safety threshold, thereby reducing the frequency of dangerous events and improving the safety and reliability of subway system operation.
[0062] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.
[0063] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the method and core idea of the present invention. At the same time, for those skilled in the art, according to the idea of the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.
Claims
1. A method for identifying passenger behavior on subway platforms in high-density passenger flow scenarios, characterized in that: The identification method comprises: Get the historical surveillance video stream of the subway platform; According to the multiple frames of images in the historical surveillance video stream that are continuous in time sequence, the passenger numbers, passenger behaviors and passenger positions of all passengers in each frame of the image are annotated to obtain historical images annotated with passenger behavior category information, and the multiple frames of images in the historical surveillance video stream that are continuous in time sequence and the historical images annotated with passenger behavior category information are used as training sets; the passenger behaviors include waiting, getting on, getting off, staying and running; the passenger positions are the coordinates of the center point of the rectangular annotated box of each passenger and the width and height of the rectangular box; The optimized YOLOv8 network is trained using the training set to obtain a passenger behavior recognition model; the optimized YOLOv8 backbone network is obtained by integrating the YOLOv8 backbone network with the passenger feature extraction model; The feature extraction model includes a dilated convolutional network, a Transformer network, and a contrastive learning loss; The optimized YOLOv8 network includes an optimized backbone network, a neck structure, and a head structure connected in sequence; the optimized backbone network is composed of the last layer of the YOLOv8 backbone network Darknet-53 connected to the feature extraction model; The last layer of the backbone network Darknet-53 of YOLOv8 is connected to the dilated convolutional network; the dilated convolutional network is connected to the Transformer network; the Transformer network is connected to the contrastive learning loss; the contrastive learning loss is connected to the neck structure; the dilated convolutional network is used to extract the compact global features of the passengers in the image output by the last layer of the backbone network Darknet-53 of YOLOv8; the Transformer network is used to perform feature decoupling on the compact global features to obtain the relevant features of each numbered passenger and the irrelevant features of the passengers; the contrastive learning loss is used to use the relevant features of the passengers as positive samples and the irrelevant features of the passengers as negative samples, to perform discriminative constraints on the relevant features of the passengers and the irrelevant features of the passengers, to separate the occluders and background noise from the global features of the passengers, and to obtain the features of the passengers; Obtain the real-time surveillance video stream of the target subway platform; Each frame of the real-time monitoring video stream is input into the passenger behavior recognition model to obtain an image marked with passenger behavior category information.
2. The method for identifying passenger behavior on subway platforms in high-density passenger flow scenarios according to claim 1 is characterized in that: The identification method further comprises: The passenger behavior recognition model is optimized by taking each frame image of the real-time monitoring video stream as input and the corresponding image labeled with passenger behavior category information as output.
3. The method for identifying passenger behavior on subway platforms in high-density passenger flow scenarios according to claim 1 is characterized in that: The identification method further comprises: According to the image marked with the passenger behavior category information, the number of passengers with the same passenger behavior within a first preset time period is counted; the first preset time period takes the current time as the end time; The number of passengers with the same passenger behavior within the first preset time period is compared with a preset first threshold to obtain a first comparison result, and a first alarm signal is generated according to the first comparison result.
4. The method for identifying passenger behavior on subway platforms in high-density passenger flow scenarios according to claim 1 is characterized in that: The identification method further comprises: According to the image marked with passenger behavior category information, the number of passengers on the platform at the current moment and the number of passengers of each type of behavior are counted; A second comparison result is obtained by comparing the number of all passengers at the current moment with a preset second threshold, and a second alarm signal is generated according to the second comparison result.
5. The method for identifying passenger behavior on subway platforms in high-density passenger flow scenarios according to claim 1 is characterized in that: The identification method further comprises: According to the image marked with the passenger behavior category information, the number of passengers with the same passenger number within the running interval between trains in a third preset time period is continuously counted; the third preset time period is a morning peak time period, an evening peak time period, an off-peak time period, a designated time period for large-scale events, or a designated time period for major holidays; the third preset time period ends at the current time; A third comparison result is obtained by comparing the number of passengers with a preset third threshold, and a third alarm signal is generated according to the third comparison result.
Citation Information
Patent Citations
Method for estimating and early warning passenger flow retained in real time in subway station
CN113343960A
Method for identifying and predicting passenger flow in rail transit station based on computer vision
CN114612860A
Gait sequence repairing and monitoring audience identity recognition method based on priori knowledge
CN115761906A
Small target detection method, storage medium and electronic equipment
CN116385773A
Lightweight target detection method and system based on YOLOv8 improvement
CN116740538A
