Passenger Flow Estimation and Display Method, System and Computer Readable Storage Medium

Abstract

The embodiments of the present invention provide a passenger flow estimation and display method, system and computer storage medium. The method performs secondary scene classification on the live video data received from the camera transmission, configures a passenger flow estimation algorithm corresponding to the scene according to the result of the scene classification, analyzes the video frames according to the passenger flow analysis algorithm, outputs the passenger flow statistics results, and determines the passenger flow congestion level at the corresponding position of the front-end camera according to the passenger flow statistics results. The passenger flow congestion level results are sent to a first terminal and a second terminal respectively for displaying the passenger flow congestion level. In some implementation processes, the method can monitor the crowd congestion level inside the subway in real time and effectively guide the flow of people.

Description

Technical Field

[0001] The embodiments of the present invention relate to, but are not limited to, the field of urban rail transit. Specifically, they relate to, but are not limited to, a passenger flow estimation and display method, system, and computer-readable storage medium. Background Art

[0002] To alleviate urban traffic congestion, more and more cities are prioritizing subway construction. Meanwhile, more and more citizens are willing to use the subway as their preferred means of public transportation. However, due to objective factors such as the large population and uneven population distribution in Chinese cities, some subway stations are overcrowded. Especially during rush hour, a large number of waiting passengers often gather inside subway stations, while passengers continue to enter the station to board. This results in an increasing number of waiting passengers inside the station and poor crowd management. This not only prolongs passenger waiting times but also creates potential safety hazards such as stampedes and fire evacuations. Therefore, how to effectively count and manage subway internal passenger flows has become a key focus of the rail transit command center network management system. In recent years, many intelligent analysis systems have been deployed, but the following shortcomings still exist:

[0003] (1) The current intelligent analysis system occupies a high amount of server resources. Each station can only connect to 16 cameras for analysis. However, for subway transfer stations prone to passenger congestion, the number of internal cameras is about 100. Connecting only 16 cameras is far from enough to truly reflect the passenger flow situation in the station.

[0004] (2) Most passenger guidance methods are simply broadcasting "There are many people in the middle of the train, please go to the ends of the train." This guidance method is relatively simple. If a large number of passengers are stranded in an emergency, the passage will be congested and passengers will not be able to know the subway operation status in advance. Manual guidance is too stressful.

[0005] (3) Due to the complex internal environment of the subway, the video classification algorithm in the intelligent analysis system usually needs to be designed specifically, and professional technicians are required to manually adapt the algorithm and delineate the ROI (Region of Interest) area during system deployment. It cannot be selected automatically, and the system deployment requires a lot of manpower and material resources.

[0006] Now it is urgent to propose a new solution to solve the problems in related technologies. Summary of the Invention

[0007] The embodiments of the present invention provide a passenger flow estimation and display method and system, which mainly solve the technical problem that the related technology cannot monitor the crowd congestion level inside the subway in real time, and the crowd flow diversion effect is unsatisfactory.

[0008] To solve the above technical problems, an embodiment of the present invention provides a passenger flow estimation and display method, including:

[0009] Receive live video data transmitted by the camera;

[0010] Performing secondary scene classification on the video data;

[0011] Configuring a passenger flow estimation algorithm corresponding to the scenario according to the result of the scenario classification;

[0012] The video frames are analyzed according to the passenger flow estimation algorithm, and the passenger flow statistics results are output. The passenger flow congestion level at the corresponding position of the front-end camera is determined based on the passenger flow statistics results, and the results of the passenger flow congestion level are sent to the first terminal and the second terminal respectively to display the passenger flow congestion degree.

[0013] An embodiment of the present invention further provides a passenger flow estimation and display system, comprising:

[0014] A front-end video acquisition module is used to acquire video data and transmit the video data to the secondary scene classification module;

[0015] A secondary scene classification module, used for performing scene classification on the video data;

[0016] An algorithm automatic configuration module, configured to configure a passenger flow analysis module corresponding to the scenario according to the result of the scenario classification;

[0017] a passenger flow analysis module, configured to analyze the video frames according to the passenger flow analysis algorithm, output passenger flow statistics results, determine the passenger flow congestion level at the position corresponding to the front-end camera according to the passenger flow statistics results, and send the passenger flow congestion level results to the first terminal and the second terminal respectively;

[0018] A first terminal passenger flow display module is configured to receive the passenger flow situation output by the passenger flow analysis module and display the situation according to the passenger flow congestion level;

[0019] The second terminal passenger flow display module is used to display the passenger flow situation output by the passenger flow analysis module.

[0020] An embodiment of the present invention further provides a computer storage medium, wherein the computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of the passenger flow estimation and display method as described above.

[0021] The beneficial effects of the present invention are:

[0022] According to the passenger flow estimation and display method, system, and computer storage medium provided by the embodiments of the present invention, the live video data received from the camera is subjected to secondary scene classification, a passenger flow estimation algorithm corresponding to the scene is configured according to the results of the scene classification, the video frames are analyzed according to the passenger flow analysis algorithm, and passenger flow statistics are output. The passenger flow congestion level at the corresponding position of the front-end camera is determined based on the passenger flow statistics, and the passenger flow congestion level results are sent to the first terminal and the second terminal respectively for displaying the passenger flow congestion level. In certain implementation processes, the crowd congestion level inside the subway can be monitored in real time, and the crowd flow can be effectively guided.

[0023] Other features and corresponding beneficial effects of the present invention are described in the latter part of the specification, and it should be understood that at least some of the beneficial effects become obvious from the description in the specification of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 Schematic diagram of the passenger flow estimation and display method according to the first embodiment of the present invention;

[0025] Figure 2 This is a schematic diagram of the secondary scene classification process of the first embodiment of the present invention;

[0026] Figure 3 Schematic diagram of the process of dividing near and far views according to the first embodiment of the present invention;

[0027] Figure 4 Schematic diagram of the passenger flow estimation and display system structure according to the second embodiment of the present invention;

[0028] Figure 5 This is a schematic diagram of the structure of a two-level scene classification system according to the second embodiment of the present invention;

[0029] Figure 6 This is a schematic diagram of the underlying feature sharing model structure of Example 2 of the present invention. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the following is a further detailed description of the embodiments of the present invention through specific implementation methods in conjunction with the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0031] Example 1:

[0032] In order to facilitate staff to monitor the crowd congestion level inside the subway in real time and effectively guide passenger flow, an embodiment of the present invention proposes a passenger flow estimation and display method.

[0033] See Figure 1 , Figure 1The following is a flow chart of a method for estimating and displaying passenger flow according to an embodiment of the present invention:

[0034] S101: Receive live video data transmitted by a camera.

[0035] In the embodiment of the present invention, a camera is arranged at the front end, and the camera will capture the on-site video of various places inside the subway station in real time, and transmit the captured video data to the secondary scene classification module.

[0036] It should be noted that the secondary scene classification module can be set on the server side. When the front end is a smart camera, it can also be set on the front end.

[0037] S102: Classify the video data into secondary scenes.

[0038] The embodiment of the present invention adopts a two-level scene classification strategy to classify the scene video data captured in real time. Specifically, Figure 2 The secondary scene classification method in an embodiment of the present invention is introduced.

[0039] S201: Perform illumination analysis preprocessing on the acquired video frames.

[0040] It should be understood that the contrast of video frames acquired at different times and locations may be different, so the video frames need to be preprocessed, that is, the video frames are subjected to illumination analysis. When the contrast of the video frame is lower than the preset contrast, the video frame is subjected to illumination enhancement processing. The processing methods include but are not limited to histogram equalization, gamma correction and other algorithms; when the contrast of the video frame is higher than the preset contrast, no processing is performed.

[0041] It should be noted that the preset contrast can be set by the system or flexibly set according to actual conditions.

[0042] S202: Perform edge detection on the video frame and extract edge contour data.

[0043] In the embodiment of the present invention, edge detection is performed on the pre-processed video frame to extract edge contour data. It should be noted that edge detection on the video frame mainly detects the edge contours of the characters in the video frame and extracts the edge contour data.

[0044] S203: Match the edge outer contour data with the first preset graphic and the second preset graphic.

[0045] In an embodiment of the present invention, a two-level scene classification strategy is employed. The first scene classification is primarily used to distinguish the types of front-end cameras, including but not limited to top-view cameras and gun cameras. It should be understood that different camera types and camera angles can result in significant differences in the image content of the acquired video frames. The first preset graphic is an outer contour graphic extracted from an image captured by the top-view camera, and the second preset graphic is an outer contour graphic extracted from an image captured by the gun camera. Templates for the first and second preset graphics are provided in the system. Images containing extracted edge outer contour data are scanned based on the templates, thereby matching the locations of graphics in the image that match the first and second preset graphics, respectively. For example, images captured by top-view cameras are mostly images of the top of a person's head, and the outline of the top of the head image is typically circular. Gun cameras, on the other hand, often capture images of the human body, and the outer contour of the head and shoulders of a person is typically Ω-shaped. Therefore, by scanning and matching the images based on the circular and Ω-shaped templates, the locations of the circles and Ω-shaped shapes in the images can be determined.

[0046] S204: Record the number of times the first preset graphic and the second preset graphic appear.

[0047] In an embodiment of the present invention, after matching the positions of the first preset graphic and the second preset graphic in the image, the matching results can be counted and the number of times the first preset graphic and the second preset graphic appear is recorded respectively, for example, the number of times a circle and an Ω shape appear is recorded respectively.

[0048] S205: Determine whether the number of times the first preset pattern appears is greater than the number of times the second preset pattern appears. If so, execute S206; if not, execute S207.

[0049] In the embodiment of the present invention, by comparing the number of occurrences of the first preset pattern and the second preset pattern, the type of the front-end camera that obtains the current video frame can be determined, that is, the first scene classification is performed.

[0050] S206: The current scene is the first scene.

[0051] In the embodiment of the present invention, the first scene is a video frame captured by a top-mounted camera.

[0052] S207: The current scene is the second scene, and the video data is classified into the second scene.

[0053] In the embodiment of the present invention, when it is determined that the current scene is the second scene, it is necessary to continue to classify the second scene. The second scene can be a video frame obtained by the gun camera. Figure 3 The second scenario classification method in the embodiment of the present invention is introduced as follows:

[0054] S301: Perform background modeling on video frames to obtain motion foreground video frames.

[0055] In an embodiment of the present invention, a mixed Gaussian model algorithm can be used to perform background modeling to obtain a motion foreground video frame. Specifically, the position of the pedestrian is represented in the video frame. For example, the position of the pedestrian is marked with a white dot. This image is called a motion foreground video frame.

[0056] S302: Obtain the proportion of foreground pixels in each row according to the motion foreground video frame.

[0057] In the embodiment of the present invention, obtaining the proportion of foreground pixels in each row is obtaining the proportion of the area marked by white dots in the video frame.

[0058] S303: Divide the video frame into a distant view area and a near view area according to the proportions.

[0059] It should be understood that the second scene classification is mainly used to divide the near and far view areas in the video frame. For example, in the far view area, the image of the person in the video frame is relatively small, and the white dot mark area accounts for a low proportion. In the near view area, the image of the person in the video frame is relatively large, and the white dot mark area accounts for a high proportion. Based on this feature, the embodiment of the present invention can determine the division position of the near view area and the far view area of the current row based on the ratio of the current row pixel duty cycle to the previous row pixel duty cycle being greater than or equal to a certain threshold. It should be noted that the threshold is usually 0.5.

[0060] S103: Configure a passenger flow analysis algorithm corresponding to the scene according to the result of the scene classification.

[0061] In this embodiment of the present invention, after scene classification is complete, a passenger flow analysis algorithm corresponding to the scene is automatically configured based on the scene classification results. For the first scenario, if the first preset pattern appears more often than the second preset pattern, the current camera is determined to be a top-view camera, and a head detection algorithm is directly configured for passenger flow analysis. For the second scenario, a head-and-shoulder detection algorithm is configured for the near-view area, and a crowd density estimation algorithm is configured for the distant view area.

[0062] S104. Analyze the video frames according to the passenger flow analysis algorithm, output the passenger flow statistics results, and determine the passenger flow congestion level at the corresponding position of the front-end camera based on the passenger flow statistics results. Send the passenger flow congestion level results to the first terminal and the second terminal respectively to display the passenger flow congestion degree.

[0063] It should be noted that in order to reduce system resource usage, the algorithm involved in passenger flow analysis in the embodiment of the present invention adopts an underlying feature sharing model. The underlying feature sharing model uses pedestrian re-identification data to extract underlying features and trains them through a convolutional neural network, so that this model can basically represent the basic information of pedestrians, and different passenger flow estimation algorithms share the parameters of this model.

[0064] In an embodiment of the present invention, the statistical results of the head detection algorithm can be directly output; the second scene classification configures two algorithms for the current video frame, a head and shoulder detection algorithm and a crowd density estimation algorithm, uses the head and shoulder detection algorithm to perform passenger flow statistics in the near-view area to obtain a first statistical result, and uses the crowd density estimation algorithm to perform passenger flow statistics in the distant view area to obtain a second statistical result, sums the first statistical result and the second statistical result, and outputs the final passenger flow statistical result, which is the passenger flow analysis result of the corresponding position of the current gun.

[0065] In the embodiments of the present invention, different passenger flow estimation algorithms share the parameters of the underlying feature sharing model, but different passenger flow estimation algorithms are performed at different network levels. For example, the head detection algorithm and the head and shoulder detection algorithm have specific differences in the detection window aspect ratio, detection window size, and detection window features. For example, when the head detection window aspect ratio is set to 1:1, the detection window selects shallow features for head detection, while when the head and shoulder detection window aspect ratio is set to 1:1.5, the detection window selects shallow features for head detection.

[0066] It should be understood that in the process of passenger flow analysis and statistics, a downsampling operation is usually performed, that is, the image is reduced. For the distant area where passenger flow statistics are performed through the crowd density estimation algorithm, a density map equal to the size of the input image needs to be output. In this case, an upsampling operation is required, that is, the reduced image is enlarged so that the output result size is consistent with the input image size, thereby outputting a density map. By summing the corresponding values of each point in the density map, the density of people contained in the input image can be obtained.

[0067] In an embodiment of the present invention, after the passenger flow statistics result is output, the passenger flow congestion level at the position corresponding to the front-end camera is determined according to the passenger flow statistics result. Specifically, the corresponding passenger flow congestion level is determined according to a preset threshold value. The passenger flow congestion can be set to four levels, namely, heavy congestion, moderate congestion, slight congestion, and no congestion. A preset threshold value is set to correspond to each level. The preset threshold value can be a range. If the output passenger flow statistics result falls within the range corresponding to a certain level, the congestion level corresponds to this passenger flow congestion level; the preset threshold value can also be a fixed value, for example, heavy congestion is 50, moderate congestion is 30, slight congestion is 20, and no congestion is 10. When the output passenger flow statistics result is 15, 15 is greater than 10 and less than 20, then the current passenger flow congestion level is no congestion. When the output passenger flow statistics result is 5, 5 is less than 10, then the current passenger flow congestion level is no congestion, and so on for other results.

[0068] In the embodiment of the present invention, after determining the passenger congestion level at the position corresponding to the front-end camera, the analyzed passenger congestion level results are sent to the first terminal and the second terminal respectively to display the passenger congestion level.

[0069] It should be noted that the first terminal is a small-screen terminal, and the second terminal is a large-screen terminal. The small-screen terminal can be a smartphone, iPad, etc. After receiving the analysis results, the corresponding areas of different camera points are mapped on the 2D map according to the passenger congestion level, and the passenger flow routes are filled with different colors, such as red for heavy congestion, orange for moderate congestion, yellow for light congestion, and green for no congestion. In this way, operation and maintenance personnel can click on the corresponding position on the screen to obtain a real-time video stream of the corresponding point, allowing station operation and maintenance personnel to have a clear understanding of the station situation at a glance and provide precise guidance for different channels and platforms.

[0070] The large-screen terminal can be used as a subway information display screen to display the passenger flow situation inside the subway in real time, making it convenient for passengers to choose their own routes, avoid congested areas, and select their subway boarding locations, thereby reducing the workload of manual guidance.

[0071] The passenger flow estimation and display method provided by the embodiment of the present invention performs secondary scene classification on the live video data transmitted by the received camera, configures the passenger flow estimation algorithm corresponding to the scene according to the result of the scene classification, and different passenger flow estimation algorithms share the parameters of the underlying feature sharing model. The video frames are analyzed according to the passenger flow analysis algorithm, and the passenger flow statistics results are output. The passenger flow congestion level at the corresponding position of the front-end camera is determined according to the passenger flow statistics results, and the analyzed passenger flow congestion level results are sent to the first terminal and the second terminal respectively for displaying the passenger flow congestion degree; by sharing the parameters of the underlying feature sharing model by different passenger flow estimation algorithms, the system resource usage is reduced. , increase the number of system access routes, and at the same time improve the overall execution strategy of the analysis algorithm. Moreover, through the secondary scene classification strategy, different types of cameras and different areas of the same camera video frame are classified, and the algorithm configuration is automated to reduce the workload of manual configuration system. The passenger flow congestion level in each area of the subway can be displayed on multiple screens. For subway operation and maintenance personnel, the passenger flow situation inside the subway can be sent to the subway operation and maintenance personnel in real time, so that the subway operation and maintenance personnel can have a clear understanding of the situation in the station, and can carry out precise guidance for different channels and platforms to improve guidance efficiency. For passengers, passengers can avoid congested areas on their own, choose the subway riding position, and reduce the workload of manual guidance.

[0072] Example 2:

[0073] The embodiment of the present invention also provides a passenger flow estimation and display system, which is used to implement at least one step of the passenger flow estimation and display method of the above embodiment. Figure 4 The system includes a video acquisition module 401, a secondary scene classification module 402, an algorithm automatic configuration module 403, a passenger flow analysis module 404, a first terminal passenger flow display module 405, and a second terminal passenger flow display module 406.

[0074] The video acquisition module 401 is used to acquire on-site video data and transmit the video data to the secondary scene classification module.

[0075] The secondary scene classification module 402 is used to classify the video data into scenes;

[0076] It should be noted that the secondary scene classification module can be set on the server side. When the front end is a smart camera, the secondary scene classification module can also be set on the front end.

[0077] The algorithm automatic configuration module 403 is used to configure the passenger flow analysis module 404 corresponding to the scene according to the result of the scene classification;

[0078] In the embodiment of the present invention, the passenger flow analysis module 404 is automatically selected by the algorithm automatic configuration module 403 according to the scene classification result. It should be noted that the algorithm automatic configuration module 403 is located on the server side.

[0079] The passenger flow analysis module 404 is configured to analyze the video frames according to the passenger flow estimation algorithm, output passenger flow statistics results, determine the passenger flow congestion level at the location corresponding to the front-end camera based on the passenger flow statistics results, and send the passenger flow congestion level results to the first terminal and the second terminal respectively;

[0080] In an embodiment of the present invention, passenger flow congestion can be set to four levels, namely, heavy congestion, moderate congestion, slight congestion and no congestion. A preset threshold is set corresponding to each level. The preset threshold can be a range. If the output passenger flow statistics result falls within the range corresponding to a certain level, the congestion level corresponds to this passenger flow congestion level; the preset threshold can also be a fixed value, for example, heavy congestion is 50, moderate congestion is 30, slight congestion is 20, and no congestion is 10. When the output passenger flow statistics result is 15, 15 is greater than 10 and less than 20, then the current passenger flow congestion level is no congestion. When the output passenger flow statistics result is 5, 5 is less than 10, then the current passenger flow congestion level is no congestion, and so on for other results.

[0081] It should be noted that the passenger flow analysis module 404 is located at the server end.

[0082] The first terminal passenger flow display module 405 is used to receive the passenger flow situation output by the passenger flow analysis module 404 and display it according to the passenger flow congestion level;

[0083] In an embodiment of the present invention, the first terminal is a small-screen terminal, which can be a smartphone, iPad, or the like. After receiving the analysis results output by the passenger flow analysis module 404, the corresponding areas of different camera points are mapped on the 2D map based on the passenger flow congestion level, and the passenger flow routes are filled with different colors, for example, red represents heavy congestion, orange represents moderate congestion, yellow represents light congestion, and green represents no congestion. In this way, operation and maintenance personnel can obtain a real-time video stream of the corresponding point by clicking on the corresponding position on the screen, allowing operation and maintenance personnel within the station to have a clear understanding of the station situation and provide precise guidance for different channels and platforms.

[0084] The second terminal passenger flow display module 406 is used to display the passenger flow situation output by the passenger flow analysis module 404 in real time.

[0085] In an embodiment of the present invention, the second terminal is a large-screen terminal, which can be a subway information release and display screen to display the passenger flow situation inside the subway in real time, making it convenient for passengers to choose their own routes, avoid congested areas, and choose their subway seats, thereby reducing the workload of manual guidance.

[0086] The embodiment of the present invention adopts a two-level scene classification strategy to realize classification of different types of cameras and different areas of the same camera video frame, automatically configure the algorithm, and reduce the workload of manual configuration system. Specifically, the two-level scene classification module 402 also includes a preprocessing module 501, an edge detection module 502, a template matching module 503, a counting module 504, a first scene classification module 505, a background modeling module 506, a duty cycle statistics module 507, and a second scene classification module 508, wherein,

[0087] The pre-processing module 501 is used to perform illumination analysis pre-processing on the acquired video frames;

[0088] It should be understood that the contrast of video frames acquired at different times and locations may be different, so the video frames need to be preprocessed, that is, the video frames are subjected to illumination analysis. When the contrast of the video frame is lower than the preset contrast, the video frame is subjected to illumination enhancement processing. The processing methods include but are not limited to histogram equalization, gamma correction and other algorithms; when the contrast of the video frame is higher than the preset contrast, no processing is performed.

[0089] It should be noted that the preset contrast can be set by the system or flexibly set according to actual conditions.

[0090] The edge detection module 502 is used to perform edge detection on the video frame and extract edge contour data.

[0091] The template matching module 503 is used to match the edge contour data obtained in advance with the first preset pattern and the second preset pattern;

[0092] In the embodiment of the present invention, a two-level scene classification strategy is employed. The subsequent first scene classification module 505 is primarily used to distinguish the types of front-end cameras, including but not limited to top-view cameras and box cameras. It should be understood that different camera types and camera angles can result in significant differences in the image content of the captured video frames. The first preset pattern is the outer contour pattern extracted from the image captured by the top-view camera, and the second preset pattern is the outer contour pattern extracted from the image captured by the box camera. Templates for the first and second preset patterns are provided in the system. Images containing the extracted edge outer contour data are scanned based on the templates, thereby matching the locations of patterns in the image that match the first and second preset patterns, respectively. For example, images captured by top-view cameras are mostly images of the top of a person's head, which typically has a circular outline. Box cameras, on the other hand, often capture images of the human body, whose head and shoulders typically have an Ω-shaped outline. Therefore, by scanning and matching the images based on the circular and Ω-shaped templates, the locations of the circles and Ω-shaped patterns in the images can be determined.

[0093] The counting module 504 is used to count the matching results and record the number of times the first preset pattern and the second preset pattern appear respectively;

[0094] In an embodiment of the present invention, after the template matching module 503 matches the positions of the first preset graphic and the second preset graphic in the image, the counting module 504 counts the matching results and records the number of times the first preset graphic and the second preset graphic appear, for example, the number of times a circle and an Ω shape appear, respectively.

[0095] The first scene classification module 505 is used to receive the number of times the first preset graphic and the second preset graphic appear output by the counting module 504. If the number of times the first preset graphic appears is greater than the number of times the second preset graphic appears, the current scene is the first scene; if the number of times the second preset graphic appears is greater than the number of times the first preset graphic appears, the current scene is the second scene, and the video data continues to be classified into the second scene.

[0096] In the embodiment of the present invention, when the current scene is the second scene, the video data is further classified into the second scene, specifically:

[0097] The background modeling module 506 is used to perform background modeling on the video frame to obtain a motion foreground video frame;

[0098] In an embodiment of the present invention, a mixed Gaussian model algorithm can be used to perform background modeling to obtain a motion foreground video frame. Specifically, the position of the pedestrian is represented in the video frame. For example, the position of the pedestrian is marked with a white dot. This image is called a motion foreground video frame.

[0099] The duty cycle statistics module 507 is used to receive a moving foreground video frame and obtain the ratio of foreground pixels in each row;

[0100] In the embodiment of the present invention, obtaining the proportion of foreground pixels in each row is obtaining the proportion of the area marked by white dots in the video frame.

[0101] The second scene classification module 508 is used to divide the video frame into a distant view area and a close view area according to the proportion;

[0102] It should be understood that the second scene classification is mainly used to divide the near and far view areas in the video frame. For example, in the far view area, the image of the person in the video frame is relatively small, and the white dot mark area accounts for a low proportion. In the near view area, the image of the person in the video frame is relatively large, and the white dot mark area accounts for a high proportion. Based on this feature, the embodiment of the present invention can determine the division position of the near view area and the far view area of the current row based on the ratio of the current row pixel duty cycle to the previous row pixel duty cycle being greater than or equal to a certain threshold. It should be noted that the threshold is usually 0.5.

[0103] In order to reduce system resource usage, the algorithm involved in the passenger flow analysis in the embodiment of the present invention adopts an underlying feature sharing model, and different passenger flow estimation algorithms share the parameters of this model.

[0104] The underlying feature sharing model includes an underlying feature extraction module 601, a head detection algorithm module 602, a head and shoulder detection algorithm module 603, a crowd density estimation algorithm module 604, an upsampling module 605, and a passenger flow counting merging module 606, wherein:

[0105] The underlying feature extraction module 601 is used to extract underlying features using pedestrian re-identification data and train them through a convolutional neural network to represent basic information of pedestrians.

[0106] The overhead detection algorithm module 602 is used to count passenger flow and output passenger flow statistics results;

[0107] In an embodiment of the present invention, when it is the first scene, it can be determined that the current camera is a top-view camera, and the algorithm automatic configuration module 403 directly and automatically configures the overhead detection algorithm to perform passenger flow analysis. The overhead detection algorithm is implemented in the overhead detection algorithm module 602, and the passenger flow statistics results are output after the passenger flow statistics are completed.

[0108] The head and shoulder detection algorithm module 603 is used to perform passenger flow statistics on the near-view area to obtain a first statistical result;

[0109] In an embodiment of the present invention, the current scene is the second scene, and the video data is classified into the second scene. The second scene classification is mainly used to divide the distant and near view areas in the video frame. After the distant and near view areas are divided, the algorithm automatic configuration module 403 configures the head and shoulder detection algorithm for the near view area to perform passenger flow statistics and obtain the first statistical result.

[0110] The upsampling module 605 is configured to perform an upsampling operation so that the size of the output video frame is the same as the size of the received video frame;

[0111] In an embodiment of the present invention, during the process of passenger flow analysis and statistics, a downsampling operation is usually performed, that is, the image is reduced. For the distant area where passenger flow statistics are performed through the crowd density estimation algorithm, a density map equal to the size of the input image needs to be output. In this case, an upsampling operation is required, that is, the reduced image is enlarged so that the output result size is consistent with the input image size.

[0112] The crowd density estimation algorithm module 604 is used to output a crowd density map, and sum up the corresponding values of the crowd density map to obtain a second statistical result.

[0113] The passenger flow counting merging module 606 is used to receive the first statistical result output by the head and shoulder detection algorithm module and the second statistical result output by the crowd density estimation algorithm module, sum the first statistical result and the second statistical result, and output the final passenger flow statistical result.

[0114] It should be noted that, in the embodiment of the present invention, different passenger flow estimation algorithms share the parameters of the underlying feature sharing model, but different passenger flow estimation algorithms are performed at different network levels. For example, the head detection algorithm module 602 and the head and shoulder detection algorithm module 603 respectively perform head and head and shoulder detection at different network levels in the underlying feature extraction module 601. The specific differences between these two modules are reflected in the detection window aspect ratio, detection window size and detection window features. For example, when the aspect ratio of the head detection window is set to 1:1, the detection window selects shallow features for head detection, while when the aspect ratio of the head and shoulder detection window is set to 1:1.5, the detection window selects shallow features for head detection.

[0115] For a detailed diagram of the secondary scene classification system structure, see Figure 5 shown.

[0116] The passenger flow estimation and display system provided by the embodiment of the present invention performs secondary scene classification on the real-time captured live video data transmitted by the video acquisition module of the front-end camera, configures the passenger flow estimation algorithm corresponding to the scene according to the result of the scene classification, and different passenger flow estimation algorithms share the parameters of the underlying feature sharing model. The video frames are analyzed according to the passenger flow analysis algorithm, and the passenger flow statistics results are output. The passenger flow congestion level of the corresponding position of the front-end camera is determined according to a preset threshold, and the analysis results are sent to the first terminal and the second terminal respectively for displaying the passenger flow congestion level. By sharing the parameters of the underlying feature sharing model by different passenger flow estimation algorithms, the system resource usage is reduced. It can increase the number of system access routes and improve the overall execution strategy of the analysis algorithm. It can also realize the classification of different types of cameras and different areas of the same camera video frames through the secondary scene classification strategy, automatically configure the algorithm, and reduce the workload of manual configuration system. It can display the passenger flow congestion level in various areas of the subway through multiple screens. For subway operation and maintenance personnel, the passenger flow situation in the subway can be sent to the subway operation and maintenance personnel in real time, so that the subway operation and maintenance personnel can have a clear understanding of the situation in the station, and can provide precise guidance for different channels and platforms to improve guidance efficiency. For passengers, it allows passengers to avoid congested areas on their own, choose the subway riding position, and reduce the workload of manual guidance.

[0117] Example 3:

[0118] The present embodiment also provides a computer-readable storage medium, which includes volatile or non-volatile, removable or non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, computer program modules or other data). Computer-readable storage media include, but are not limited to, RAM (Random Access Memory), ROM (Read-Only Memory), EEPROM (Electrically Erasable Programmable read only memory), flash memory or other memory technology, CD-ROM (Compact Disc Read-Only Memory), digital versatile disk (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer.

[0119] The computer-readable storage medium in this embodiment can be used to store one or more computer programs, and the one or more computer programs stored therein can be executed by a processor to implement at least one step of the passenger flow estimation and display method in the above-mentioned embodiment 1.

[0120] It should be understood that, in some cases, at least one step shown or described may be performed in an order different from that described in the above embodiments.

[0121] It can be seen that those skilled in the art should understand that all or some of the steps in the method disclosed above, and the functional modules / units in the system can be implemented as software (which can be implemented using computer program code executable by a computing device), firmware, hardware, and appropriate combinations thereof. In hardware implementations, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component can have multiple functions, or a function or step can be performed by several physical components in cooperation. Some or all physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit.

[0122] In addition, it is well known to those skilled in the art that communication media generally contain computer-readable instructions, data structures, computer program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media. Therefore, the present invention is not limited to any specific hardware and software combination.

[0123] The above content is a further detailed description of the embodiments of the present invention in conjunction with specific implementation methods, and the specific implementation of the present invention cannot be considered to be limited to these descriptions. For those skilled in the art of the present invention, without departing from the concept of the present invention, several simple deductions or substitutions can be made, which should be considered to fall within the scope of protection of the present invention.

Claims

1. A method for estimating and displaying passenger flow, comprising: Receive video data transmitted by the camera; Performing secondary scene classification on the video data; Configuring a passenger flow analysis algorithm corresponding to the scenario based on the result of the scenario classification; Analyze the video frames according to the passenger flow analysis algorithm, output passenger flow statistics results, determine the passenger flow congestion level at the position corresponding to the camera according to the passenger flow statistics results, and send the passenger flow congestion level results to the first terminal and the second terminal respectively for displaying the passenger flow congestion degree; The performing secondary scene classification on the video data includes: Perform illumination analysis preprocessing on the acquired video frames; Performing edge detection on the video frame and extracting edge contour data; Matching the edge outer contour data with a first preset graphic and a second preset graphic; Recording the number of times the first preset graphic and the second preset graphic appear to perform a first scene classification, where the first scene classification is used to distinguish the type of camera corresponding to the video frame; If the number of times the first preset graphic appears is greater than the number of times the second preset graphic appears, the current scene is the first scene; If the second preset graphic appears more times than the first preset graphic appears, the current scene is the second scene, and the video data is classified into the second scene. The first scene and the second scene are scenes of video frames shot by different types of cameras, respectively. The second scene classification is used to divide the video frames into near and far view areas.

2. The passenger flow estimation and display method according to claim 1, characterized in that: The performing a second scene classification on the video data comprises: Performing background modeling on the video frame to obtain a motion foreground video frame; According to the motion foreground video frame, obtaining the proportion of foreground pixels in each row; The video frame is divided into a distant view area and a near view area according to the proportion.

3. The passenger flow estimation and display method according to claim 1, wherein: When the current scene is the first scene, the passenger flow analysis algorithm corresponding to the scene is an overhead detection algorithm.

4. The passenger flow estimation and display method according to claim 2, characterized in that: Configuring a passenger flow analysis algorithm corresponding to a scenario based on the results of the scenario classification includes: The near view area is configured with a head and shoulder detection algorithm; The distant view area is configured with a crowd density estimation algorithm.

5. The passenger flow estimation and display method according to claim 4, characterized in that: Analyzing the video frames according to the passenger flow analysis algorithm and outputting passenger flow statistics results includes: Performing passenger flow statistics in the near-view area using a head-and-shoulder detection algorithm to obtain a first statistical result; Performing passenger flow statistics in the distant area using a crowd density estimation algorithm to obtain a second statistical result; The first statistical result and the second statistical result are summed to obtain a final passenger flow statistical result.

6. The passenger flow estimation and display method according to claim 5, characterized in that: The passenger flow statistics of the distant area are performed using a crowd density estimation algorithm to obtain a second statistical result, which includes: Performing a downsampling operation on the video frame to collect feature information of a crowd in the video frame; Performing crowd density analysis based on the characteristic information; An upsampling operation is performed on the video frame, a crowd density map is output, and a second statistical result is obtained.

7. The passenger flow estimation and display method according to any one of claims 1 to 6, characterized in that: The passenger flow analysis algorithm uses an underlying feature sharing model to perform passenger flow statistics.

8. A passenger flow estimation and display system, comprising: A video acquisition module, configured to acquire video data and transmit the video data to a secondary scene classification module; A secondary scene classification module, configured to classify the video data into scenes; An algorithm automatic configuration module, configured to configure a passenger flow analysis algorithm corresponding to the scenario according to the result of the scenario classification; a passenger flow analysis module, configured to analyze the video frames according to the passenger flow analysis algorithm, output passenger flow statistics results, determine the passenger flow congestion level at the position corresponding to the camera according to the passenger flow statistics results, and send the passenger flow congestion level results to the first terminal and the second terminal respectively; A first terminal passenger flow display module is configured to receive the passenger flow information output by the passenger flow analysis module and display the information based on the passenger flow congestion level; The second terminal passenger flow display module is used to display the passenger flow situation output by the passenger flow analysis module; The secondary scene classification module includes: A preprocessing module is used to perform illumination analysis preprocessing on the acquired video frames; An edge detection module is used to perform edge detection on the video frame and extract edge contour data; A template matching module, configured to match the edge contour data with a first preset pattern and a second preset pattern; A counting module, configured to count the matching results and record the number of times the first preset pattern and the second preset pattern appear respectively; a first scene classification module, configured to receive the number of occurrences of the first preset pattern and the second preset pattern output by the counting module to perform a first scene classification, wherein the first scene classification is used to distinguish the type of camera corresponding to the video frame; if the number of occurrences of the first preset pattern is greater than the number of occurrences of the second preset pattern, the current scene is the first scene; if the number of occurrences of the second preset pattern is greater than the number of occurrences of the first preset pattern, the current scene is the second scene, wherein the first scene and the second scene are scenes of video frames captured by different types of cameras, respectively; The second scene classification module is used to continue to perform second scene classification on the video data if the current scene is the second scene. The second scene classification is used to divide the video frame into near and far scene areas.

9. The passenger flow estimation and display system according to claim 8, characterized in that: The secondary scene classification module also includes: A background modeling module, configured to perform background modeling on the video frame to obtain a motion foreground video frame; A duty cycle statistics module is used to receive the motion foreground video frame and obtain the ratio of foreground pixels in each row; The second scene classification module is used to divide the video frame into a distant view area and a near view area according to the proportion.

10. The passenger flow estimation and display system according to claim 9, characterized in that: The system further comprises: The underlying feature extraction module is used to extract underlying features using pedestrian re-identification data and train them through a convolutional neural network to represent basic pedestrian information; Overhead detection algorithm module, used to count passenger flow and output passenger flow statistics results; a head-and-shoulder detection algorithm module, configured to perform passenger flow statistics on the near-view area to obtain a first statistical result; an upsampling module, configured to perform an upsampling operation so that the size of the output video frame is the same as the size of the received video frame; a crowd density estimation algorithm module, configured to output a crowd density map, and sum corresponding values of the crowd density map to obtain a second statistical result; The passenger flow counting merging module is used to receive the first statistical result output by the head and shoulder detection algorithm module and the second statistical result output by the crowd density estimation algorithm module, sum the first statistical result and the second statistical result, and output a final passenger flow statistical result.

11. A computer-readable storage medium, characterized in that The computer-readable storage medium stores one or more computer programs, and the one or more computer programs can be executed by one or more processors to implement the steps of the passenger flow estimation and display method according to any one of claims 1 to 7.

Images