Passenger flow statistical method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application disclose a passenger flow statistical method and device, electronic equipment and a storage medium. The method comprises: acquiring at least one first to-be-recognized image obtained by capturing a front door of a bus, at least one second to-be-recognized image obtained by capturing a rear door of the bus, and a plurality of third to-be-recognized images obtained by capturing a bus platform of a bus stop where the bus stops; determining a first passenger flow in the bus according to the at least one first to-be-recognized image and the at least one second to-be-recognized image; determining a second passenger flow of the bus platform according to the plurality of third to-be-recognized images; and determining a passenger flow of the bus stop where the bus stops according to the first passenger flow in the bus and the second passenger flow of the bus platform. The embodiments of the present application are beneficial to improving the accuracy of passenger flow statistics.

Description

Technical Field

[0001] The present application relates to the field of image recognition technology, and specifically to a passenger flow counting method, device, electronic device and storage medium. Background Art

[0002] Counting passenger flow is very important for some businesses or governments. For example, for bus operation, the passenger flow at each station can be counted, and then the traffic management department can dynamically plan traffic routes based on the passenger flow at each station, providing intelligent bus operation.

[0003] At present, the statistical method for bus passenger flow is generally to obtain a depth image when passengers board the bus, denoise the depth image to obtain an effective depth map, perform background modeling based on the effective depth map to obtain a foreground image, perform image segmentation on the foreground image to obtain a segmented image, obtain a pedestrian profile based on the segmented image, perform pedestrian head detection based on the pedestrian profile to obtain a detection frame, use the detection frame as the current tracking frame, and then use a tracking method to track the tracking frame to obtain the passenger flow at the bus stop in real time.

[0004] However, this statistical method only takes into account the number of passengers on the bus. The way of counting passenger flow is relatively simple and does not truly count the actual passenger flow at each station, resulting in relatively low statistical accuracy. Summary of the Invention

[0005] The embodiments of the present application provide a passenger flow counting method, device, electronic device, and storage medium, which improve the statistical accuracy of passenger flow by associating and matching human body frames and human heads.

[0006] In a first aspect, an embodiment of the present application provides a method for counting passenger flow, including:

[0007] Acquire at least one frame of a first image to be recognized obtained by capturing a front door of a bus, at least one frame of a second image to be recognized obtained by capturing a rear door of the bus, and multiple frames of a third image to be recognized obtained by capturing a bus stop at a bus stop;

[0008] determining a first passenger flow in the bus according to the at least one frame of the first image to be recognized and the at least one frame of the second image to be recognized;

[0009] determining a second passenger flow of the bus stop according to the multiple frames of the third to-be-recognized images;

[0010] The passenger flow of the bus stop where the bus stops is determined according to the first passenger flow in the bus and the second passenger flow at the bus stop.

[0011] In a second aspect, an embodiment of the present application provides a passenger flow counting device, comprising:

[0012] a transceiver unit configured to respectively acquire at least one frame of a first image to be recognized obtained by capturing a front door of a bus, at least one frame of a second image to be recognized obtained by capturing a rear door of the bus, and multiple frames of a third image to be recognized obtained by capturing a bus platform at a bus stop where the bus stops;

[0013] a processing unit, configured to determine a first passenger flow in the bus based on the at least one frame of the first image to be recognized and the at least one frame of the second image to be recognized;

[0014] determining a second passenger flow of the bus stop according to the multiple frames of the third to-be-recognized images;

[0015] The passenger flow of the bus stop where the bus stops is determined according to the first passenger flow in the bus and the second passenger flow at the bus stop.

[0016] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor, the processor being connected to a memory, the memory being used to store a computer program, the processor being used to execute the computer program stored in the memory, so that the electronic device executes the method described in the first aspect.

[0017] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program enables a computer to execute the method described in the first aspect.

[0018] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer is operable to enable the computer to execute the method described in the first aspect.

[0019] The implementation of the embodiments of the present application has the following beneficial effects:

[0020] It can be seen that in the embodiment of the present application, at least one frame of the first image to be identified and at least one frame of the second image to be identified are obtained by capturing the front and rear doors of the bus, and multiple frames of the third image to be identified are obtained by capturing the bus platform of the bus stop where the bus is parked; then, the first passenger flow in the bus is determined based on the at least one frame of the first image to be identified and the at least one frame of the second image to be identified, and the second passenger flow at the bus stop is determined based on the multiple frames of the third image to be identified; finally, the total passenger flow at the bus stop is determined based on the first passenger flow and the second passenger flow. This way of counting the passenger flow not only considers the passengers on the bus, but also counts the second passenger flow waiting for the bus at the bus stop, thereby making the passenger flow count at the bus stop more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0022] Figure 1 A schematic diagram of the architecture of a passenger flow statistics system provided in an embodiment of the present application;

[0023] Figure 2 A flow chart of a passenger flow statistics method provided in an embodiment of the present application;

[0024] Figure 3 A flowchart of another passenger flow counting method provided in an embodiment of the present application;

[0025] Figure 4 A block diagram of the functional units of a passenger flow counting device provided in an embodiment of the present application;

[0026] Figure 5 A schematic diagram of the structure of a passenger flow counting device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0028] The terms "first," "second," "third," and "fourth," etc., in the specification and claims of this application and the accompanying drawings are used to distinguish between different objects, rather than to describe a specific order. In addition, the terms "including," "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or elements is not limited to the listed steps or elements, but may optionally include steps or elements not listed, or may optionally include other steps or elements inherent to the process, method, product, or apparatus.

[0029] References herein to "embodiments" mean that a particular feature, result, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0030] See Figure 1 , Figure 1 This is a schematic diagram of the architecture of a passenger flow counting system provided in an embodiment of the present application. The passenger flow counting system includes a passenger flow counting device 10 and at least three image acquisition devices 20 (three image acquisition devices 20 are used as an example in this application). The passenger flow counting device 10 and the three image acquisition devices maintain a communication connection.

[0031] like Figure 1 As shown, three image acquisition devices 20 are installed on the bus. The passenger flow counting device 10 can also be installed on the bus or separately from the image acquisition device, which is not limited in this application. Specifically, one image acquisition device 20 is installed at the front door of the bus to capture passengers getting on and off the bus from the front door, obtain at least one frame of the first image to be identified, and send the captured at least one frame of the first image to be identified to the passenger flow counting device 10; one image acquisition device 20 is installed at the back of the bus to capture passengers getting on and off the bus from the back of the bus to capture at least one frame of the second image to be identified, and send the captured at least one frame of the second image to be identified to the passenger flow counting device 10; one image acquisition device is installed on the roof of the bus to capture passengers at the bus stop to capture multiple frames of the third image to be identified, and send the multiple frames of the third image to be identified to the passenger flow counting device 10;

[0032] Further, the passenger flow counting device 10 determines the first passenger flow in the bus according to the at least one first to-be-recognized image and the at least one second to-be-recognized image, and determines the second passenger flow at the bus stop according to the multiple third to-be-recognized images; finally, the passenger flow at the bus stop where the bus stops is determined according to the first passenger flow in the bus and the second passenger flow at the bus stop.

[0033] It can be seen that in the embodiment of the present application, the image capturing device 20 respectively captures the front door and the back door of the bus to obtain the at least one first to-be-recognized image and the at least one second to-be-recognized image, and the passenger flow counting device 10 determines the first passenger flow in the bus according to the at least one first to-be-recognized image and the at least one second to-be-recognized image. In addition, the image capturing device 20 also captures the bus stop where the bus stops to obtain the multiple third to-be-recognized images, and determines the second passenger flow at the bus stop according to the multiple third to-be-recognized images. Finally, the passenger flow at the bus stop is determined according to the first passenger flow and the second passenger flow. This way of counting passenger flow not only considers the passengers on the bus, but also counts the second passenger flow waiting at the bus stop, so that the passenger flow at the bus stop is more accurate.

[0034] It should be understood that the image capturing device involved in the present application can be an analog video camera, a digital video camera, an SDTV camera, an HD camera, a CCD camera, a COMS camera, or other devices with image capturing function, which are not limited in the present application.

[0035] Referring to Figure 2 , Figure 2 a flowchart of a passenger flow counting method provided by the embodiment of the present application. The method is applied to a passenger flow counting device. The method includes the following steps:

[0036] 201: The passenger flow counting device respectively acquires the at least one first to-be-recognized image obtained by capturing the front door of the bus, the at least one second to-be-recognized image obtained by capturing the back door of the bus, and the multiple third to-be-recognized images obtained by capturing the bus stop of the bus stop where the bus stops.

[0037] For example, as shown in the passenger flow counting system, Figure 1 the image capturing device arranged at the front door of the bus captures the image to obtain the at least one first to-be-recognized image, the image capturing device arranged at the back of the bus captures the image to obtain the at least one second to-be-recognized image, and the image capturing device arranged at the top of the bus captures the image to obtain the multiple third to-be-recognized images.

[0038] 202: The passenger flow counting device determines a first passenger flow in the bus based on the at least one frame of the first image to be recognized and the at least one frame of the second image to be recognized.

[0039] Exemplarily, the passenger flow counting device performs human body detection on each of at least one first image to be identified frame to obtain at least one first human body frame in each first image to be identified frame. Specifically, target detection (human body detection) is performed on the first image to be identified to obtain multiple candidate human body frames. Non-maximum suppression is then performed on the candidate human body frames to obtain the at least one first human body frame. Target detection on the first image to be identified can be implemented using a commonly used human body detection network, such as a convolutional neural network (CNN), such as a V-net, U-net, or fast-CNN.

[0040] Then, a human head association matching is performed on each first human frame in each frame of the first image to be recognized to obtain a human head associated with each first human frame in each frame of the first image to be recognized; based on the human head associated with each first human frame, the number of people getting on and off the bus corresponding to each frame of the first image to be recognized is determined. For example, based on the head orientation of the human head associated with each first human frame, it is determined whether the passenger in each human frame is a passenger getting on or off the bus. In this way, it is possible to determine the passengers getting on and off the bus in each frame of the first image to be recognized, and then obtain the number of people getting on and off the bus corresponding to each frame of the first image to be recognized; similarly, human body detection is performed on each second image to be recognized in the at least one frame of the second image to be recognized to obtain at least one second human frame corresponding to each frame of the second image to be recognized; then, a human head association is performed on each second human frame in the at least one second human frame to obtain a human head associated with each second human frame; based on the human head associated with each second human frame, the number of people getting on and off the bus corresponding to each frame of the second image to be recognized is determined; finally, based on the number of people getting on and off the bus corresponding to each frame of the first image to be recognized and the number of people getting on and off the bus corresponding to each frame of the second image to be recognized, the first passenger flow in the bus (i.e., the number of passengers on the bus) is determined. Specifically, the passengers getting on and off the bus corresponding to each frame of the first image to be identified in the at least one frame of the first image to be identified are deduplicated to obtain the number of people getting on and off the bus from the front door. Similarly, the passengers getting on and off the bus corresponding to each frame of the second image to be identified in the at least one frame of the second image to be identified are deduplicated to obtain the number of people getting on and off the bus from the rear door. Finally, the number of people getting on and off the bus is determined based on the number of people getting on and off the bus from the front door and the number of people getting on and off the bus from the rear door. The first passenger flow in the bus is determined based on the number of people getting on and off the bus at the bus stop and the number of people remaining in the bus after leaving the previous bus stop.

[0041] The process of performing human-head association matching is described below using a human body frame A, wherein the human body frame A is any one of the at least one first human body frame or any one of the at least one second human body frame.

[0042] In one embodiment of the present application, the relative height of a human body frame A in a first image to be identified can be determined; based on the relative height of the human body frame A in the first image to be identified and the corresponding scaling ratio of the first image to be identified, the actual height of the human body in the human body frame A can be determined; based on the actual height of the human body in the human body frame A, the target position of the human head associated with the human body frame A in the first image to be identified can be determined; and the human head in the target position in the human body frame A can be used as the human head associated with the human body frame A.

[0043] In one embodiment of the present application, a first relative area of ​​the human body frame A in the first image to be identified, that is, the area of ​​the human body frame A in the first image to be identified, is determined; and a second relative area of ​​each head in the human body frame in the first image to be identified is determined. In addition, in the case where the human head is occluded, the entire human head is completed based on the unoccluded part. For example, the pixel points belonging to the human head in the human body frame are determined, and the pixel points belonging to the human head are formed into an area. When the shape of the area does not match the standard shape corresponding to the human head, it is determined that the human head in the human body frame is occluded, and the human head is completed according to the shape of the area formed by the pixel points belonging to the human head in the human body frame and the standard shape corresponding to the human head, that is, it is completed to the standard shape; then, the area occupied by the completed entire human head in the first image to be identified is used as the second relative area of ​​the human head in the first image to be identified; then, the ratio between the second relative area of ​​each head in the human body frame and the first relative area of ​​the human body frame A is determined to obtain the ratio corresponding to each head in the human body frame A; the head whose ratio is within the preset ratio range is used as the head associated with the human body frame A, that is, the head associated with the human body frame A is determined by coordinating the proportions between the head and the human body.

[0044] In one embodiment of the present application, skin color detection is performed on the human body in the human body frame A to obtain the skin color type of the human body in the human body frame A, that is, the exposed area of ​​the human body framed in the human body frame is determined, and the average value of the pixels in the pixel points in the area is used as the skin color type of the human body; then, skin color detection is performed on each face in the human body frame to obtain the skin color type corresponding to each face, and the average value of the pixels in each face is also obtained as the skin color type of each face; finally, the difference between the skin color type of the face and the skin color type of the human body (that is, the difference in pixels) is within a preset difference range, that is, the associated face of the human body frame A is determined through skin color matching.

[0045] In actual application, passengers getting on and off the bus may be crowded, so that multiple persons are framed in each person frame, and the heads framed in two person frames may be repeated heads. If each head in each person frame is regarded as an independent passenger without association and matching, the passenger flow is repeatedly counted, and the passenger flow counting is not accurate enough. It can be seen that, in the embodiment of the present application, the person and the head are associated, so that only one head in each person frame is counted each time, and the problem of repeated counting is avoided, thereby improving the counting accuracy of the passenger flow.

[0046] 203: The passenger flow counting device determines the second passenger flow of the bus stop according to the plurality of third to-be-recognized images.

[0047] For example, the direction of movement and the speed of movement of each pedestrian on the bus stop are determined according to the plurality of third to-be-recognized images, and the second passenger flow of the bus stop is determined according to the direction of movement and the speed of movement of each pedestrian.

[0048] Specifically, the face direction of each pedestrian on the bus stop is determined according to the plurality of third to-be-recognized images, and the face direction of each pedestrian is taken as the direction of movement of each pedestrian. The movement distance of each pedestrian in the process of capturing any two adjacent second to-be-recognized images in the plurality of third to-be-recognized images is determined, for example, the first relative distance between each pedestrian and a reference object in the previous second to-be-recognized image of the two second to-be-recognized images and the second relative distance between each pedestrian and the reference object in the next second to-be-recognized image of the two second to-be-recognized images are determined, and the absolute difference between the first relative distance and the second relative distance is determined. The movement distance of each pedestrian in the process of capturing any two adjacent second to-be-recognized images is determined according to the absolute difference and the scaling ratio of the third to-be-recognized image, wherein the reference object can be a bus stop board of the bus stop.

[0049] For example, a pedestrian whose direction of movement is parallel to the bus stop and whose speed of movement is greater than a threshold value (for example, a pedestrian passing by the bus stop) is taken as a pedestrian passing by the bus stop, a pedestrian whose direction of movement is perpendicular to the bus stop (for example, a pedestrian waiting for a bus at the bus stop) or whose speed of movement is less than or equal to a threshold value (for example, a pedestrian pacing back and forth at the bus stop) is taken as a passenger waiting for a bus at the bus stop, and the passenger waiting for a bus at the bus stop is taken as the second passenger flow of the bus stop.

[0050] It can be seen that, in the embodiment of the present application, the speed of movement and the direction of movement are comprehensively judged, so that some pedestrians passing by the bus stop are excluded, and thus the counted second passenger flow is more accurate.

[0051] 204: The passenger flow counting device determines the passenger flow of the bus stop where the bus stops based on the first passenger flow in the bus and the second passenger flow at the bus stop.

[0052] Exemplarily, the passenger flow counting device takes the sum of the first passenger flow and the second passenger flow as the passenger flow of the bus stop.

[0053] It can be seen that in the embodiment of the present application, at least one frame of the first image to be identified and at least one frame of the second image to be identified are obtained by capturing the front and rear doors of the bus, and multiple frames of the third image to be identified are obtained by capturing the bus platform of the bus stop where the bus is parked; then, the first passenger flow in the bus is determined based on the at least one frame of the first image to be identified and the at least one frame of the second image to be identified, and the second passenger flow at the bus stop is determined based on the multiple frames of the third image to be identified; finally, the total passenger flow at the bus stop is determined based on the first passenger flow and the second passenger flow. This way of counting the passenger flow not only considers the passengers on the bus, but also counts the second passenger flow waiting for the bus at the bus stop, thereby making the passenger flow count at the bus stop more accurate.

[0054] See Figure 3 , Figure 3 A flow chart of another passenger flow statistics method provided in an embodiment of the present application. The method uses a passenger flow statistics device. Figure 2 The same contents as those in the embodiment shown are not described again here. The method of this embodiment includes the following steps:

[0055] 301: The passenger flow counting device obtains at least one frame of a first image to be recognized obtained by capturing a front door of a bus, at least one frame of a second image to be recognized obtained by capturing a rear door of the bus, and multiple frames of a third image to be recognized obtained by capturing a bus platform at a bus stop where the bus is parked.

[0056] 302: The passenger flow counting device determines a first passenger flow in the bus based on the at least one frame of the first image to be recognized and the at least one frame of the second image to be recognized.

[0057] 303: The passenger flow counting device determines a second passenger flow at the bus stop based on the multiple frames of the third image to be recognized.

[0058] 304: The passenger flow counting device determines the passenger flow of the bus stop where the bus stops based on the first passenger flow in the bus and the second passenger flow at the bus stop.

[0059] 305: The passenger flow counting device sends the passenger flow of the bus stops where the bus stops to the bus dispatching center, so that the bus dispatching center can plan the bus routes according to the passenger flow of each bus stop.

[0060] For example, the passenger flow statistics device can obtain the passenger flow of each bus stopping at any bus stop at each stop time, and then upload the passenger flow of the stop to the bus dispatching center. In this way, the bus dispatching center can obtain the passenger flow of any stop based on the passenger flow of each bus stopping at any bus stop at each stop time, obtain the number of buses passing through the bus stop, and determine the total carrying capacity of these buses. If the total carrying capacity is greater than the passenger flow of the stop, the bus with the longest route among these buses will be determined to cancel its stop at the bus stop, thereby increasing the running speed of the bus.

[0061] It can be seen that in the embodiment of the present application, at least one frame of the first image to be identified and at least one frame of the second image to be identified are obtained by capturing the front and rear doors of the bus, and multiple frames of the third image to be identified are obtained by capturing the bus platform of the bus stop where the bus is parked; then, the first passenger flow in the bus is determined based on the at least one frame of the first image to be identified and the at least one frame of the second image to be identified, and the second passenger flow at the bus stop is determined based on the multiple frames of the third image to be identified; finally, the total passenger flow at the bus stop is determined based on the first passenger flow and the second passenger flow. This way of counting the passenger flow not only considers the passengers on the bus, but also counts the second passenger flow waiting for the bus at the bus stop, thereby making the passenger flow count at the bus stop more accurate; in addition, the total passenger flow of each bus stop is reported to the bus dispatching center, so as to optimize the bus operation route and improve the efficiency of bus operation.

[0062] See Figure 4 , Figure 4 The present invention provides a functional block diagram of a passenger flow counting device. The passenger flow counting device 400 includes a transceiver unit 401 and a processing unit 402, wherein:

[0063] The transceiver unit 401 is configured to respectively acquire at least one frame of a first image to be recognized obtained by capturing a front door of a bus, at least one frame of a second image to be recognized obtained by capturing a rear door of the bus, and multiple frames of a third image to be recognized obtained by capturing a bus platform at a bus stop where the bus is parked;

[0064] The processing unit 402 is configured to determine a first passenger flow in the bus based on the at least one frame of the first image to be recognized and the at least one frame of the second image to be recognized;

[0065] determining a second passenger flow of the bus stop according to the multiple frames of the third to-be-recognized images;

[0066] The passenger flow of the bus stop where the bus stops is determined according to the first passenger flow in the bus and the second passenger flow at the bus stop.

[0067] In some possible implementations, in determining the first passenger flow in the bus based on the at least one frame of the first image to be recognized and the at least one frame of the second image to be recognized, the processing unit 402 is specifically configured to:

[0068] Performing human body detection on each of the at least one frame of the first image to be recognized to obtain at least one first human body frame corresponding to each of the first image to be recognized;

[0069] Performing human head association matching on each first human body frame in the at least one first human body frame to obtain a human head associated with each first human body frame;

[0070] Determining the number of people getting on and off the bus corresponding to each frame of the first image to be recognized based on the human heads associated with each first human body frame;

[0071] Performing human body detection on each second image to be recognized in the at least one frame of the second image to be recognized to obtain at least one second human body frame corresponding to each second image to be recognized;

[0072] Performing human head association matching on each second human body frame in the at least one second human body frame to obtain a human head associated with each second human body frame;

[0073] Determining the number of people getting on and off the bus corresponding to each frame of the second image to be recognized based on the human heads associated with each second human body frame;

[0074] A first passenger flow in the bus is determined according to the number of people getting on and off the bus corresponding to each frame of the first image to be recognized and the number of people getting on and off the bus corresponding to each frame of the second image to be recognized.

[0075] In some possible implementations, in performing human-head association matching, the processing unit 402 is specifically configured to:

[0076] Determining the relative height of the human body frame A in the first image to be recognized;

[0077] determining the actual height of the person in the person frame A according to the relative height of the person frame A in the first image to be recognized and the scaling ratio corresponding to the first image to be recognized;

[0078] determining, based on the actual height of the person in the person frame A, a target position of a head associated with the person frame A in the first image to be recognized;

[0079] The human head at the target position in the human body frame A is used as the human head associated with the human body frame A;

[0080] The human body frame A is any one of the at least one first human body frame or any one of the at least one second human body frame.

[0081] In some possible implementations, in performing human-head association matching, the processing unit 402 is specifically configured to:

[0082] Determining a first relative area of ​​a human body frame A in the first image to be recognized;

[0083] Determining a second relative area of ​​each head in the human body frame A in the first image to be recognized;

[0084] Determine a ratio between a second relative area of ​​each head in the human body frame A and a first relative area of ​​the human body frame A to obtain a ratio corresponding to each head in the human body frame A;

[0085] A human head with a ratio within a preset ratio range is regarded as a human head associated with the human body frame A;

[0086] The human body frame A is any one of the at least one first human body frame or any one of the at least one second human body frame.

[0087] In some possible implementations, in determining the second passenger flow of the bus stop based on the multiple frames of the third image to be recognized, the processing unit 402 is specifically configured to:

[0088] determining the travel direction and travel speed of each pedestrian on the bus stop based on the multiple frames of the third to-be-recognized images;

[0089] A second passenger flow of the bus stop is determined according to the travel direction and travel speed of each pedestrian at the bus stop.

[0090] In some possible implementations, in determining the travel direction and travel speed of each pedestrian at the bus stop based on the multiple frames of the third to-be-recognized image, the processing unit 402 is specifically configured to:

[0091] determining, based on the multiple frames of the third to-be-recognized images, a facial orientation of each pedestrian at the bus stop, and using the facial orientation of each pedestrian as a travel direction of each pedestrian;

[0092] determining, based on any two adjacent frames of the second to-be-recognized images in the plurality of frames of the third to-be-recognized images, a distance traveled by each pedestrian in the process of capturing the any two adjacent frames of the second to-be-recognized images;

[0093] determining a travel speed of each pedestrian in the process of capturing the any two adjacent frames of the second image to be recognized according to a travel distance and a capture interval of each pedestrian in the process of capturing the any two adjacent frames of the second image to be recognized;

[0094] The average of the travel speeds during the process of any two adjacent frames of the second image to be recognized is calculated to obtain the travel speed of each pedestrian in the bus stop.

[0095] In some possible implementations, in determining the second passenger flow at the bus stop based on the travel direction and travel speed of each pedestrian at the bus stop, the processing unit 402 is specifically configured to:

[0096] Pedestrians whose travel direction is parallel to the bus stop and whose travel speed is greater than a threshold are regarded as pedestrians passing through the bus stop;

[0097] Treat pedestrians whose travel direction is perpendicular to the bus stop or whose travel speed is less than or equal to the threshold as passengers waiting for the bus at the bus stop;

[0098] The passengers waiting for the bus at the bus stop are regarded as the second passenger flow of the bus stop.

[0099] See Figure 5 , Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 5 As shown, the electronic device 500 includes a transceiver 501, a processor 502, and a memory 503. These are connected via a bus 504. The memory 503 is used to store computer programs and data, and can transmit the data stored in the memory 503 to the processor 502.

[0100] The processor 502 is configured to read the computer program in the memory 503 and perform the following operations:

[0101] Controlling the transceiver 501 to obtain at least one frame of a first image to be recognized obtained by capturing a front door of a bus, at least one frame of a second image to be recognized obtained by capturing a rear door of the bus, and multiple frames of a third image to be recognized obtained by capturing a bus platform at a bus stop where the bus stops;

[0102] determining a first passenger flow in the bus according to the at least one frame of the first image to be recognized and the at least one frame of the second image to be recognized;

[0103] determining a second passenger flow of the bus stop according to the multiple frames of the third to-be-recognized images;

[0104] The passenger flow of the bus stop where the bus stops is determined according to the first passenger flow in the bus and the second passenger flow at the bus stop.

[0105] In some possible implementations, in determining the first passenger flow in the bus based on the at least one frame of the first image to be recognized and the at least one frame of the second image to be recognized, the processor 502 is specifically configured to perform the following operations:

[0106] Performing human body detection on each of the at least one frame of the first image to be recognized to obtain at least one first human body frame corresponding to each of the first image to be recognized;

[0107] Performing human head association matching on each first human body frame in the at least one first human body frame to obtain a human head associated with each first human body frame;

[0108] Determining the number of people getting on and off the bus corresponding to each frame of the first image to be recognized based on the human heads associated with each first human body frame;

[0109] Performing human body detection on each second image to be recognized in the at least one frame of the second image to be recognized to obtain at least one second human body frame corresponding to each second image to be recognized;

[0110] Performing human head association matching on each second human body frame in the at least one second human body frame to obtain a human head associated with each second human body frame;

[0111] Determining the number of people getting on and off the bus corresponding to each frame of the second image to be recognized based on the human heads associated with each second human body frame;

[0112] A first passenger flow in the bus is determined according to the number of people getting on and off the bus corresponding to each frame of the first image to be recognized and the number of people getting on and off the bus corresponding to each frame of the second image to be recognized.

[0113] In some possible implementations, in performing human-head association matching, the processor 502 is specifically configured to perform the following operations:

[0114] Determining the relative height of the human frame A in the first image to be recognized;

[0115] determining the actual height of the person in the person frame A according to the relative height of the person frame A in the first image to be recognized and the scaling ratio corresponding to the first image to be recognized;

[0116] determining, based on the actual height of the person in the person frame A, a target position of a head associated with the person frame A in the first image to be recognized;

[0117] The human head at the target position in the human body frame A is used as the human head associated with the human body frame A;

[0118] The human body frame A is any one of the at least one first human body frame or any one of the at least one second human body frame.

[0119] In some possible implementations, in performing human-head association matching, the processor 502 is specifically configured to perform the following operations:

[0120] Determining a first relative area of ​​a human body frame A in the first image to be recognized;

[0121] Determining a second relative area of ​​each head in the human body frame A in the first image to be recognized;

[0122] Determine a ratio between a second relative area of ​​each head in the human body frame A and a first relative area of ​​the human body frame A to obtain a ratio corresponding to each head in the human body frame A;

[0123] A human head with a ratio within a preset ratio range is regarded as a human head associated with the human body frame A;

[0124] The human body frame A is any one of the at least one first human body frame or any one of the at least one second human body frame.

[0125] In some possible implementations, in determining the second passenger flow at the bus stop based on the multiple frames of the third image to be recognized, the processor 502 is specifically configured to perform the following operations:

[0126] determining the travel direction and travel speed of each pedestrian on the bus stop based on the multiple frames of the third to-be-recognized images;

[0127] A second passenger flow of the bus stop is determined according to the travel direction and travel speed of each pedestrian at the bus stop.

[0128] In some possible implementations, in determining the travel direction and travel speed of each pedestrian at the bus stop based on the multiple frames of the third to-be-recognized image, the processor 502 is specifically configured to perform the following operations:

[0129] determining, based on the multiple frames of the third to-be-recognized images, a facial orientation of each pedestrian at the bus stop, and using the facial orientation of each pedestrian as a travel direction of each pedestrian;

[0130] determining, based on any two adjacent frames of the second to-be-recognized images in the plurality of frames of the third to-be-recognized images, a distance traveled by each pedestrian in the process of capturing the any two adjacent frames of the second to-be-recognized images;

[0131] determining a travel speed of each pedestrian in the process of capturing the any two adjacent frames of the second image to be recognized according to a travel distance and a capture interval of each pedestrian in the process of capturing the any two adjacent frames of the second image to be recognized;

[0132] The average of the travel speeds during the process of any two adjacent frames of the second image to be recognized is calculated to obtain the travel speed of each pedestrian in the bus stop.

[0133] In some possible implementations, in determining the second passenger flow at the bus stop based on the travel direction and travel speed of each pedestrian at the bus stop, the processor 502 is specifically configured to perform the following operations:

[0134] Pedestrians whose travel direction is parallel to the bus stop and whose travel speed is greater than a threshold are regarded as pedestrians passing through the bus stop;

[0135] Treat pedestrians whose travel direction is perpendicular to the bus stop or whose travel speed is less than or equal to the threshold as passengers waiting for the bus at the bus stop;

[0136] The passengers waiting for the bus at the bus stop are regarded as the second passenger flow of the bus stop.

[0137] Specifically, the transceiver 501 may be Figure 4 The transceiver unit 401 of the passenger flow counting device 400 of the embodiment described above, the processor 502 may be Figure 4 The processing unit 402 of the passenger flow counting device 400 of the embodiment described above.

[0138] It should be understood that the electronic devices in this application may include smartphones (such as Android phones, iOS phones, Windows Phone phones, etc.), tablet computers, PDAs, laptops, mobile Internet devices (MIDs) or wearable devices. The above electronic devices are only examples and are not exhaustive, including but not limited to the above electronic devices. In actual applications, the above electronic devices may also include: smart car terminals, computer equipment, etc.

[0139] The embodiment of the present application further provides a computer readable storage medium, which stores a computer program. The computer program is executed by a processor to implement part or all steps of any one of the crowd flow counting methods described in the above method embodiments.

[0140] The embodiment of the present application further provides a computer program product, which comprises a non-transitory computer readable storage medium storing a computer program. The computer program is operable to cause a computer to perform part or all steps of any one of the crowd flow counting methods described in the above method embodiments.

[0141] It should be noted that, for the above-mentioned method embodiments, in order to simply describe, they are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited to the action sequence described, because according to the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present application.

[0142] In the above embodiments, the description of each embodiment is focused on, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0143] In several embodiments provided by the present application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are only schematic. The division of the units is only a logical function division. There can be another division manner in actual implementation, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical or other forms.

[0144] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0145] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of software program modules.

[0146] The integrated unit, if implemented in the form of a software program module and sold or used as an independent product, can be stored in a computer readable memory. Based on such understanding, the technical solutions of the present application essentially or the part of the prior art that contributes to the present application or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a memory and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the method described in the various embodiments of the present application. The aforementioned memory includes: a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0147] A person of ordinary skill in the art can understand that all or part of the steps of the various methods of the above embodiments can be completed by instructing the relevant hardware through a program, and the program can be stored in a computer readable memory, and the memory can include: a flash disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0148] The embodiments of the present application are described in detail above, and the principles and implementation manners of the present application are described by applying specific examples. The above description of the embodiments is only used to help understand the method of the present application and its core idea; at the same time, for those skilled in the art, according to the idea of the present application, the specific implementation manner and application range will be changed, and the above description of the embodiments is not used to limit the present application.

Claims

1. A passenger flow statistics method, characterized in that: include: Acquire at least one frame of a first image to be recognized obtained by capturing a front door of a bus, at least one frame of a second image to be recognized obtained by capturing a rear door of the bus, and multiple frames of a third image to be recognized obtained by capturing a bus stop at a bus stop; determining a first passenger flow in the bus according to the at least one frame of the first image to be recognized and the at least one frame of the second image to be recognized; Determining a second passenger flow at the bus stop according to the plurality of frames of the third to-be-recognized images includes: Determine the facial orientation of each pedestrian at the bus stop based on the multiple frames of the third image to be recognized, and use the facial orientation of each pedestrian as the travel direction of each pedestrian; determine the travel distance of each pedestrian in the process of capturing the arbitrary two adjacent frames of the second image to be recognized based on any two adjacent frames of the second image to be recognized in the multiple frames of the third image to be recognized; determine the travel speed of each pedestrian in the process of capturing the arbitrary two adjacent frames of the second image to be recognized based on the travel distance and capture interval of each pedestrian in the process of capturing the arbitrary two adjacent frames of the second image to be recognized; average the travel speeds in the process of capturing the arbitrary two adjacent frames of the second image to be recognized to obtain the travel speed of each pedestrian at the bus stop; Pedestrians whose travel direction is parallel to the bus stop and whose travel speed is greater than a threshold are considered pedestrians passing through the bus stop; pedestrians whose travel direction is perpendicular to the bus stop or whose travel speed is less than or equal to the threshold are considered passengers waiting for the bus at the bus stop; and passengers waiting for the bus at the bus stop are considered the second passenger flow of the bus stop; The passenger flow of the bus stop where the bus stops is determined according to the first passenger flow in the bus and the second passenger flow at the bus stop.

2. The method according to claim 1, characterized in that The determining of a first passenger flow in the bus according to the at least one frame of the first image to be recognized and the at least one frame of the second image to be recognized includes: Performing human body detection on each of the at least one frame of the first image to be recognized to obtain at least one first human body frame corresponding to each of the first image to be recognized; Performing human head association matching on each first human body frame in the at least one first human body frame to obtain a human head associated with each first human body frame; Determining the number of people getting on and off the bus corresponding to each frame of the first image to be recognized based on the human heads associated with each first human body frame; Performing human body detection on each second image to be recognized in the at least one frame of the second image to be recognized to obtain at least one second human body frame corresponding to each second image to be recognized; Performing human head association matching on each second human body frame in the at least one second human body frame to obtain a human head associated with each second human body frame; Determining the number of people getting on and off the bus corresponding to each frame of the second image to be recognized based on the human heads associated with each second human body frame; A first passenger flow in the bus is determined according to the number of people getting on and off the bus corresponding to each frame of the first image to be recognized and the number of people getting on and off the bus corresponding to each frame of the second image to be recognized.

3. The method according to claim 2, characterized in that The human body and head association matching includes: Determining the relative height of the human body frame A in the first image to be recognized; determining the actual height of the person in the person frame A according to the relative height of the person frame A in the first image to be recognized and the scaling ratio corresponding to the first image to be recognized; determining, based on the actual height of the person in the person frame A, a target position of a head associated with the person frame A in the first image to be recognized; The human head at the target position in the human body frame A is used as the human head associated with the human body frame A; The human body frame A is any one of the at least one first human body frame or any one of the at least one second human body frame.

4. The method according to claim 2, characterized in that The human body and head association matching includes: Determining a first relative area of ​​a human body frame A in the first image to be recognized; Determining a second relative area of ​​each head in the human body frame A in the first image to be recognized; Determine a ratio between a second relative area of ​​each head in the human body frame A and a first relative area of ​​the human body frame A to obtain a ratio corresponding to each head in the human body frame A; The head with a ratio within a preset ratio range is regarded as the head associated with the human body frame A; The human body frame A is any one of the at least one first human body frame or any one of the at least one second human body frame.

5. A passenger flow counting device, characterized in that: include: a transceiver unit configured to respectively acquire at least one frame of a first image to be recognized obtained by capturing a front door of a bus, at least one frame of a second image to be recognized obtained by capturing a rear door of the bus, and multiple frames of a third image to be recognized obtained by capturing a bus platform at a bus stop where the bus stops; a processing unit, configured to determine a first passenger flow in the bus based on the at least one frame of the first image to be recognized and the at least one frame of the second image to be recognized; determining a second passenger flow of the bus stop according to the multiple frames of the third to-be-recognized images; The passenger flow of the bus stop where the bus stops is determined according to the first passenger flow in the bus and the second passenger flow at the bus stop.

6. An electronic device, characterized in that: include: A processor and a memory, the processor is connected to the memory, the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the electronic device performs the method according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the method according to any one of claims 1 to 4.

Images