Passenger flow statistical method and system based on deep learning, and storage medium

Abstract

The invention discloses a passenger flow statistics method and system based on deep learning and a storage medium, and relates to the technical field of passenger flow statistics. The method comprises the following steps: step 1, different monitoring ends are set up in a passenger flow circulation area, passenger flow features are extracted and analyzed, and a feature database is constructed based on an analysis result; 2, configuring an implementation scheme database, wherein the implementation scheme database comprises a first implementation scheme, a second implementation scheme and a third implementation scheme; performing statistics on problems generated in the implementation process through the implementation scheme database, generating corresponding solutions and constructing a solution scheme database; 3, constructing a passenger flow statistical model, and carrying out statistics on the passenger flow; collecting statistical data generated in the statistical process in a centralized manner, and establishing a collection and storage library for storage; meanwhile, problems generated in the implementation process are collected in a centralized mode, and feedback adjustment is conducted based on the problems in the later period.

Description

Technical Field

[0001] This invention relates to the field of passenger flow statistics technology, specifically to a passenger flow statistics method, system, and storage medium based on deep learning. Background Technology

[0002] Currently, passenger flow statistics refer to the process of systematically collecting, recording, analyzing, and generating reports on the number of people, their behavioral characteristics, and movement directions within a specific area and time period using various technological means. However, existing passenger flow statistics methods have limitations in terms of feature definition: on the one hand, they cannot comprehensively cover all types of passenger flow characteristics generated during the statistical process; on the other hand, most conventional statistical methods can only be limited to basic human characteristics, making it difficult to adapt to the statistical needs of complex scenarios, ultimately leading to discrepancies between statistical results and actual conditions. Summary of the Invention

[0003] The purpose of this invention is to provide a deep learning-based passenger flow statistics method, system, and storage medium to solve the problems mentioned in the background art.

[0004] To achieve the above objectives, the present invention provides the following technical solution: a deep learning-based passenger flow statistics method, system, and storage medium, comprising the following steps: Step 1: By setting up different monitoring terminals in the passenger flow area, passenger flow characteristics are extracted. After the extraction is completed, the characteristics are analyzed, and a characteristic database is established. Step 2: Configure the implementation scheme database, which includes a first implementation scheme, a second implementation scheme, and a third implementation scheme. The implementation scheme database is used to statistically analyze the problems that arise during the implementation process. After the statistics are completed, solutions are generated and a solution database is formed. Step 3: Build a passenger flow statistics model, collect passenger flow statistics, collect the generated statistical data in a centralized manner, and store the data in a collection repository. Collect any problems that arise during implementation and make adjustments based on feedback later.

[0005] Preferably, the passenger flow characteristics include appearance characteristics, motion characteristics, and aggregation characteristics, wherein the appearance characteristics include global characteristics, local characteristics, and semantic attribute characteristics, the motion characteristics include location and speed, and motion trajectory, and the aggregation characteristics include flow characteristics, density characteristics, and behavior and time sequence; The feature extraction method for the appearance includes: (1) extracting global features: extracting the depth feature vector from the entire detection box, using a convolutional neural network pre-trained on a large pedestrian re-identification dataset to extract a high-dimensional vector, the high-dimensional vector encoding the appearance information of the pedestrian's head and shoulders; (2) extracting local features: focusing on specific local areas of the pedestrian's body, using a pose estimation model to first locate the pedestrian's key points, and then extracting the features of these areas respectively, or horizontally dividing the pedestrian image into several stripes and extracting features respectively; (3) extracting semantic attribute features: identifying the high-level semantic attributes of the pedestrian, using a classification network to determine the pedestrian's gender, age group, whether they are wearing a hat, color of their upper garment, style of their lower garment, and whether they are carrying a backpack, etc. The motion feature extraction method includes extracting position and velocity: representing the position using the coordinates of the center point of the target detection box; and extracting the motion trajectory: connecting consecutive position points of a target over a period of time to form a trajectory. The aggregation feature extraction method includes extracting traffic features: inbound / outbound customer flow; total customer flow, total number of people in a specific time period; throughput, customer flow passing through a certain cross section per unit time; conversion rate, a core indicator of the retail industry; extracting density features: real-time density of the area, number of people per unit area; average density, average density over a period of time; extracting behavior and time sequence: average dwell time; browsing routes; spatiotemporal distribution map.

[0006] Preferably, in step one, when analyzing the features, association rule analysis, predictive analysis, cluster analysis, and root cause analysis are employed. Specifically, these analysis methods include: association rule analysis: uncovering implicit relationships between customer flow features; predictive analysis: using a long-term memory network model to predict short-term customer flow; cluster analysis: using unsupervised learning algorithms to group customers and developing strategies for different groups; and root cause analysis: when abnormal data occurs, drilling down to investigate the causes and combining external data for multi-dimensional cross-analysis to pinpoint the root cause of the problem.

[0007] Preferably, in step two, the problems generated during the implementation process are statistically analyzed. These problems include occlusion, lighting variation, the balance between model accuracy and speed, and perspective effect. Specifically, the problems include: occlusion: when people are densely packed, they occlude each other, resulting in incomplete or missing detection boxes; lighting: changes in day / night, cloudy / sunny days, and indoor lighting lead to significant differences in image quality; the balance between model accuracy and speed: high-precision models are slow and cannot process multiple video streams in real time; and perspective effect: when the camera is looking down, people in the distance appear smaller, while people nearby appear larger, leading to detection and counting errors.

[0008] Preferably, the specific steps for constructing the solution database in step two include: occlusion solutions: utilizing the appearance feature re-identification capability of the Deep Simple Online Real-Time Tracking algorithm and employing representative points to detect key points; illumination variation solutions: using data augmentation, incorporating data under illumination conditions during model training, and employing features insensitive to illumination; solutions for balancing model accuracy and speed: selecting lightweight models, using model pruning, and deploying them on edge computing devices; and perspective effect solutions: performing perspective transformation or camera calibration to map image coordinates to real-world coordinates, thereby weighting or normalizing people in different positions.

[0009] Preferably, the specific steps for constructing the passenger flow statistics model in step three include: using a monocular vision method based on traditional image processing, employing a single camera to identify human figures and counting them using virtual lines or regions; using a monocular vision method based on deep learning, employing the YOLO deep learning object detection model to accurately identify each person in the video frame, then assigning a unique ID to each person using a simple online real-time tracking algorithm to form a trajectory, and finally determining the direction of entry and exit and counting using virtual lines; using a binocular vision / 3D stereo vision method, employing two cameras to calculate the depth information of the target through parallax; and using a 3D depth camera method, directly acquiring the depth map of the scene by emitting infrared speckle or light pulses and receiving the returned light, without relying on visible light.

[0010] Preferably, the data collection and storage scheme in step three is as follows: First, a collection repository dedicated to storing passenger flow statistics is constructed, and then the centrally collected passenger flow statistics are transmitted to the collection repository in real time; the collection repository adopts a management mode of classified archiving and secure storage to manage the passenger flow statistics.

[0011] Preferably, in step three, during the feedback process, analysis is performed by an intelligent analysis system and by experts. After dual analysis, feedback adjustment schemes are generated, namely a first feedback adjustment scheme, a second feedback adjustment scheme, and a third feedback adjustment scheme. The first feedback adjustment scheme has the greatest feedback adjustment force, while the third feedback adjustment scheme has the least feedback adjustment force.

[0012] A deep learning-based passenger flow statistics system includes a feature extraction and analysis unit, an implementation plan formulation unit, and a passenger flow statistics unit. The feature extraction and analysis unit extracts passenger flow features by setting up different monitoring terminals in the passenger flow area, analyzes the features after extraction, and establishes a feature database after analysis. The implementation scheme formulation unit configures an implementation scheme database, which includes a first implementation scheme, a second implementation scheme, and a third implementation scheme. The implementation scheme database is used to statistically analyze the problems that arise during the implementation process. Once the statistics are completed, solutions are generated and a solution database is formed. The passenger flow statistics unit constructs a passenger flow statistics model to count passenger flow. During the statistics process, the generated statistical data is collected centrally and stored in a collection repository. Problems encountered during implementation are collected centrally and feedback is provided for adjustments later.

[0013] A storage medium having a computer program stored thereon, which, when executed by a processor, implements the deep learning-based passenger flow statistics method described above.

[0014] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. Before conducting passenger flow statistics, this invention comprehensively limits the extracted passenger flow characteristics, specifically from the characteristics of the human body and the dynamic characteristics of movement, thereby improving the accuracy of the subsequent statistical process, increasing the efficiency of the response process, and avoiding the situation where the flow of people is too large and dense during the statistical process, which would lead to errors between the final statistics and the actual number of people. 2. This invention can statistically analyze various problems that arise during implementation in real time, and adjust the implementation plan through a feedback mechanism to improve the timeliness of the implementation plan. Attached Figure Description

[0015] Figure 1 A method flowchart is provided for embodiments of the present invention; Figure 2 This is a system structure block diagram provided for an embodiment of the present invention. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] Example 1 Please see Figure 1 This invention provides a technical solution: a passenger flow statistics method based on deep learning, comprising the following steps: Step 1: By setting up different monitoring terminals in the passenger flow area, passenger flow characteristics are extracted. After the extraction is completed, the characteristics are analyzed, and a characteristic database is established. Step 2: Configure the implementation scheme database, which includes a first implementation scheme, a second implementation scheme, and a third implementation scheme. The implementation scheme database is used to statistically analyze the problems that arise during the implementation process. Once the statistics are completed, solutions are generated and a solution database is formed. Step 3: Build a passenger flow statistics model to collect passenger flow statistics. During the statistics process, collect the generated statistical data centrally and store it in a collection repository. Collect any problems that arise during implementation and make adjustments based on feedback later.

[0018] In step one, the passenger flow characteristics include appearance characteristics, motion characteristics, and aggregation characteristics. The appearance characteristics include global characteristics, local characteristics, and semantic attribute characteristics. The motion characteristics include location, speed, and motion trajectory. The aggregation characteristics include flow characteristics, density characteristics, and behavior and time-series correlation. (1) The method for extracting global features is as follows: the depth feature vector extracted from the entire detection box (the whole body of the pedestrian) is used to extract a high-dimensional vector (such as 256 or 512 dimensions) using a convolutional neural network (CNN) pre-trained on a large pedestrian re-identification (Re-ID) dataset. The high-dimensional vector encodes the head and shoulder appearance information of the pedestrian. (2) The method for extracting local features is as follows: focus on specific local areas of the pedestrian's body, use the pose estimation model to first locate the pedestrian's key points (head, shoulders, elbows, knees, etc.), and then extract the features of these areas respectively; or divide the pedestrian image horizontally into several stripes and extract the features respectively. (3) The method for extracting semantic attribute features is to identify the high-level semantic attributes of pedestrians and use a classification network to determine the pedestrian's gender, age group, whether they are wearing a hat, the color of their upper garment, the style of their lower garment, and whether they are carrying a backpack, etc. (4) The method for extracting position and velocity is as follows: the position is represented by the center point coordinates (x, y) of the target detection box. In consecutive frames, the position of the next frame is predicted by Kalman Filter, and its velocity (vx, vy) and acceleration are estimated. (5) The method for extracting motion trajectory is to connect the continuous position points of a target over a period of time to form a trajectory, which itself is a powerful temporal feature; (6) Extracting traffic characteristics includes: ① Customer flow entering the store / customer flow leaving the store, obtained by counting through virtual lines; ② Total customer flow, the total number of people in a specific time period; ③ Throughput, the number of customers passing through a certain section in a unit of time (e.g., per hour); ④ Conversion rate, (number of customers who made a purchase / number of customers entering the store) * 100%, a core indicator of the retail industry. (7) Extracted density features include: ① Real-time density of the area and the number of people per unit area (people / square meter), which are key indicators for public safety early warning; ② Average density and the average density over a period of time. (8) Extracting behavior related to time sequence specifically includes: ① Average dwell time (total dwell time of all customers / total number of customers), reflecting the attractiveness of the area; ② Walking route, by analyzing the trajectory of a large number of pedestrians, extracting the hot movement line, which is the movement line where the passenger flow exceeds the hotness threshold; ③ Spatiotemporal distribution map: analyzing the distribution of passenger flow at different times of the day (such as morning peak and midday trough).

[0019] In step one, when analyzing the features, association rule analysis, predictive analysis, cluster analysis, and root cause analysis are employed. Specifically, association rule analysis involves uncovering implicit relationships between customer flow characteristics (e.g., discovering that "when the weather index is greater than 30 degrees Celsius, the customer flow in the ice cream area of ​​the mall is negatively correlated with the customer flow in the clothing area on the second floor," thus suggesting the installation of more customer attraction facilities on the first floor during hot weather). Predictive analysis uses an LSTM (Long Short-Term Memory) model to predict future short-term (next hour, tomorrow) or long-term (next month, holidays) customer flow, which is then applied to the advance allocation of manpower, materials, and security resources. Cluster analysis uses unsupervised learning algorithms (such as K-Means). DBSCAN segments customers, potentially uncovering unexpected customer groups such as "fast shoppers," "deep experience seekers," and "social shoppers." Strategies are then developed for each group. Root cause analysis specifically involves: when abnormal data occurs (such as a sudden drop in customer traffic on a particular day), drill-down analysis is used to investigate the cause, combined with multi-dimensional cross-analysis of external data (weather, holidays, competitor information) to pinpoint the root cause of the problem.

[0020] In step two, the problems encountered during implementation are statistically analyzed. These problems include occlusion, lighting variations, the balance between model accuracy and speed, and perspective effects. Specifically, occlusion occurs when people are densely packed, causing them to occlude each other, resulting in incomplete or missing detection boxes. Lighting variations occur due to differences in image quality caused by changes in day / night, cloudy / sunny weather, and indoor lighting. The balance between model accuracy and speed occurs because high-precision models (such as Faster R-CNN) are slow and cannot process multiple video streams in real time. Perspective effects occur when the camera is looking down, making distant people appear smaller and nearby people appear larger, leading to detection and counting errors.

[0021] The specific steps for constructing the solution database in step two are as follows; (1) Occlusion solution: Utilize DeepSORT's appearance feature re-identification capability and use RepPoints to detect key points; (2) Lighting variation solutions: Use data for data augmentation (such as adjusting brightness and contrast), add data under lighting conditions during model training, and use features that are not sensitive to lighting (such as edges and shapes). (3) Solution for balancing model accuracy and speed: Select a lightweight model (such as YOLO-lite, MobileNet-SSD), use model pruning, and deploy it on edge computing devices (such as NVIDIA Jetson); (4) Perspective effect solution: Perform perspective transformation or camera calibration to map image coordinates to real-world coordinates, thereby weighting or normalizing people in different positions and improving counting accuracy.

[0022] The specific steps for constructing the passenger flow statistics model in step three are as follows: (1) Statistical analysis is performed using a monocular vision method based on traditional image processing, using a single camera to identify human figures and counting them through virtual lines or regions; (2) Statistical analysis is performed using a monocular vision method based on deep learning. Deep learning target detection models such as YOLO and SSD are used to accurately identify each person in the video frame. Then, a unique ID is assigned to each person through tracking algorithms such as DeepSORT to form a trajectory. Finally, the direction of entry and exit is determined and counted by virtual lines. (3) Statistical analysis is performed using binocular vision / 3D stereo vision methods, mimicking the human eye, and two cameras are used to calculate the depth information of the target through parallax; (4) 3D depth camera method to perform statistics, directly obtain the depth map of the scene by emitting infrared speckle (structured light) or light pulse (ToF) and receiving the returned light, without relying on visible light.

[0023] In step three, during the feedback process, analysis is conducted through an intelligent analysis system and by experts. After dual analysis, feedback adjustment plans are formulated, namely, a first feedback adjustment plan, a second feedback adjustment plan, and a third feedback adjustment plan. The first feedback adjustment plan has the greatest adjustment effect, while the third feedback adjustment plan has the least adjustment effect.

[0024] Example 2 A deep learning-based passenger flow statistics system includes a feature extraction and analysis unit, an implementation plan formulation unit, and a passenger flow statistics unit. The feature extraction and analysis unit extracts passenger flow features by setting up different monitoring terminals in the passenger flow area, analyzes the features after extraction, and establishes a feature database after analysis. The implementation scheme formulation unit configures an implementation scheme database, and the implementation scheme includes a first implementation scheme, a second implementation scheme, and a third implementation scheme. The implementation scheme database is used to statistically analyze the problems that arise during the implementation process. When the statistics are completed, solutions are generated and a solution database is formed. The passenger flow statistics unit constructs a passenger flow statistics model to count passenger flow. During the statistics process, the generated statistical data is collected centrally and stored in a collection repository. Problems encountered during implementation are collected centrally and feedback is provided for adjustments later.

[0025] Example 3 A storage medium having a computer program stored thereon, which, when executed by a processor, implements the deep learning-based passenger flow statistics method described above.

[0026] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0027] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A deep learning-based passenger flow statistics method, characterized in that, Includes the following steps: Step 1: By setting up different monitoring terminals in the passenger flow area, extract and analyze passenger flow characteristics, and build a feature database based on the analysis results; Step 2: Configure the implementation scheme database, which includes a first implementation scheme, a second implementation scheme, and a third implementation scheme; use the implementation scheme database to statistically analyze the problems that arise during implementation, generate corresponding solutions, and build a solution database; Step 3: Build a passenger flow statistics model and perform passenger flow statistics; Collect statistical data generated during the statistical process in a centralized manner and establish a collection repository for storage; At the same time, problems arising during the implementation process are collected centrally, and feedback and adjustments are made based on these problems later.

2. The deep learning-based passenger flow statistics method according to claim 1, characterized in that: The passenger flow characteristics include appearance characteristics, motion characteristics, and aggregation characteristics; wherein, the appearance characteristics include global characteristics, local characteristics, and semantic attribute characteristics, the motion characteristics include location and speed and motion trajectory, and the aggregation characteristics include flow characteristics, density characteristics, and behavior and time sequence; The method for extracting the appearance features includes: global features are depth feature vectors extracted from the entire detection box, and a high-dimensional vector encoding the appearance information of the pedestrian's head and shoulders is extracted using a convolutional neural network pre-trained on a large pedestrian re-identification dataset; local features are extracted by focusing on specific local regions of the pedestrian's body, specifically: using a pose estimation model to first locate the pedestrian's key points and then extracting the features of each region separately, or by horizontally segmenting the pedestrian image into several stripes and then extracting the features separately; semantic attribute features are used to identify the high-level semantic attributes of the pedestrian, and a classification network is used to determine the pedestrian's gender, age group, whether they are wearing a hat, the color of their upper garment, the style of their lower garment, and whether they are carrying a backpack, etc. The method for extracting motion features includes: position and velocity are represented by the coordinates of the center point of the target detection box; the motion trajectory is a trajectory formed by connecting the continuous position points of the target over a period of time. The extraction methods for the aggregated features include: traffic features including inbound customer flow, outbound customer flow, total customer flow, total number of people in a specific time period, throughput, customer flow passing through a certain cross section per unit time, conversion rate, and core indicators of the retail industry; density features including real-time density of the area, number of people per unit area, average density, and average density over a period of time; and behavioral and temporal features including average dwell time, shopping routes, and spatiotemporal distribution maps.

3. The deep learning-based passenger flow statistics method according to claim 2, characterized in that: In step one, when analyzing the features, association rule analysis, predictive analysis, cluster analysis, and root cause analysis are employed; the specific analysis methods are as follows: Association rule analysis: uncovering implicit relationships between customer flow characteristics; predictive analysis: using a long short-term memory network model to predict future short-term or long-term customer flow; cluster analysis: using unsupervised learning algorithms to group customers and develop corresponding strategies for different groups; root cause analysis: when abnormal data occurs, drilling down to investigate the cause and combining external data for multi-dimensional cross-analysis to locate the root cause of the problem.

4. The deep learning-based passenger flow statistics method according to claim 3, characterized in that: The problems encountered during the implementation process in step two include occlusion, lighting variations, the balance between model accuracy and speed, and perspective effects; each problem is specifically as follows: Occlusion issues: When people are densely packed, they may occlude each other, resulting in incomplete or missing detection boxes; Illumination issues: Changes in lighting conditions, such as daytime, nighttime, cloudy days, sunny days, and indoor lighting, lead to significant differences in image quality. The trade-off between model accuracy and speed: high-precision models run slowly and cannot process multiple video streams in real time; Perspective effect problem: When the camera shoots from above, people in the distance appear smaller than people in the foreground, causing discrepancies in detection and counting.

5. The deep learning-based passenger flow statistics method according to claim 4, characterized in that: The solution database in step two is constructed using the following specific method: Solution to the occlusion problem: Utilize the appearance feature re-identification capability of the Deep Simple Online Real-Time Tracking Algorithm and use representative points to detect key points; Solution to the problem of changing lighting conditions: Through data augmentation, data under lighting conditions are added during model training, and features that are insensitive to lighting conditions are adopted; A solution to balance model accuracy and speed: Select lightweight models, combine them with model pruning techniques, and deploy them on edge computing devices; Solution to the perspective effect problem: Map the image coordinates to real-world coordinates through perspective transformation or camera calibration, and perform weighted or normalized processing on people in different positions.

6. The deep learning-based passenger flow statistics method according to claim 5, characterized in that: The passenger flow statistics model in step three is constructed using the following method: Monocular visual statistical methods based on traditional image processing: using a single camera to identify human figures and counting them through virtual lines or designated areas; A deep learning-based monocular vision statistical method: The YOLO deep learning object detection model is used to accurately identify each person in the video frame. A deep simple online real-time tracking algorithm is used to assign a unique ID to each person and form a motion trajectory. Finally, the direction of people entering and exiting is determined by virtual lines and the counting is completed. Binocular vision or 3D stereoscopic vision statistical method: Using two cameras, the depth information of the target is calculated through parallax to realize passenger flow statistics; 3D depth camera statistical method: By emitting infrared speckle or light pulses and receiving the returned light, the depth map of the scene can be directly obtained, and the flow of people can be counted without relying on visible light.

7. The deep learning-based passenger flow statistics method according to claim 6, characterized in that: In step three, a dedicated collection and storage repository for storing passenger flow statistics is first established, and then the centrally collected passenger flow statistics are transmitted to the collection and storage repository in real time. The collection repository manages the passenger flow statistics through categorized archiving and secure storage.

8. The deep learning-based passenger flow statistics method according to claim 7, characterized in that: The implementation feedback process in step three involves first analyzing passenger flow characteristics and generating a feedback adjustment plan based on the analysis results. The feedback adjustment plan includes a first feedback adjustment plan, a second feedback adjustment plan, and a third feedback adjustment plan. Among them, the first feedback adjustment plan has the greatest adjustment effect, and the third feedback adjustment plan has the smallest adjustment effect.

9. A passenger flow statistics system based on deep learning, characterized in that: The passenger flow statistics system is applicable to the deep learning-based passenger flow statistics method described in claim 8, and includes a feature extraction and analysis unit, an implementation plan formulation unit, and a passenger flow statistics unit. The functions of each unit are as follows: Feature extraction and analysis unit: Different monitoring terminals are set up in the passenger flow area to extract and analyze passenger flow characteristics, and a feature database is built based on the analysis results; Implementation plan formulation unit: Configures an implementation plan database, which includes a first implementation plan, a second implementation plan, and a third implementation plan; uses this database to statistically analyze problems arising during implementation, generates corresponding solutions, and constructs a solution database. Passenger flow statistics unit: Constructs a passenger flow statistics model to collect passenger flow statistics; Collect statistical data generated during the statistical process in a centralized manner and establish a collection repository for storage; At the same time, problems arising during the implementation process will be collected centrally, and feedback and adjustments will be made based on these problems later.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the deep learning-based passenger flow statistics method as described in any one of claims 1-8.

Images