Passenger Flow Statistics Method, Device, Electronic Device and Storage Medium

Abstract

An embodiment of the present application discloses a passenger flow statistics method, apparatus, electronic device, and storage medium. The method includes: collecting at least two frames of to-be-recognized images of pedestrians through at least two image acquisition devices, where the shooting angles of the at least two image acquisition devices are different, and the at least two image acquisition devices correspond to the at least two frames of to-be-recognized images one by one, and the pedestrians are any pedestrians entering a target venue; determining the identity information of the pedestrians according to the at least two frames of to-be-recognized images; and determining the passenger flow of the target venue according to the identity information of the pedestrians. The embodiment of the present application is beneficial to improving the statistical accuracy of the passenger flow.

Description

Technical Field

[0001] This application relates to the field of image recognition technology, and particularly relates to a passenger flow statistics method, device, electronic device and storage medium. Background Art

[0002] Counting the passenger flow is relatively important for some merchants or the government. For example, for bus operation, counting the passenger flow at each station, and then the traffic control department can dynamically plan the traffic routes based on the passenger flow at each station to provide the intelligence of bus operation; for the developers of shopping malls, counting the passenger flow of each shopping mall, comparing the passenger flow with the transaction volume, using the passenger flow statistical counter to count the specific number of people in the shopping mall for 24 hours a day, and then analyzing the daily transaction ratio through the passenger flow and the transaction ratio on the specific day to provide an important indicator for evaluating the rationality of rent and an important parameter for attracting investment.

[0003] Currently, most of the ways to identify the passenger flow are to set corresponding image acquisition devices at the entrances and exits, and then, through image recognition, obtain the number of people entering and leaving, and count the passenger flow based on the number of people entering and leaving. This statistical method is relatively single, and the accuracy of the counted passenger flow is relatively low. Summary of the Invention

[0004] The embodiments of this application provide a passenger flow statistics method, device, electronic device and storage medium, which improve the statistical accuracy of the passenger flow by identifying the identity information of pedestrians.

[0005] In a first aspect, the embodiments of this application provide a passenger flow statistics method, including:

[0006] Collect at least two frames of to-be-recognized images of pedestrians through at least two image acquisition devices, where the shooting angles of the at least two image acquisition devices are different, and the at least two image acquisition devices correspond to the at least two frames of to-be-recognized images one by one, and the pedestrian is any pedestrian entering the target place;

[0007] Determine the identity information of the pedestrian according to the at least two frames of to-be-recognized images;

[0008] Determine the passenger flow of the target place according to the identity information of the pedestrian.

[0009] In a second aspect, the embodiments of this application provide a passenger flow statistics device, including:

[0010] An acquisition unit, configured to collect at least two frames of to-be-recognized images of pedestrians through at least two image acquisition devices, where the shooting angles of the at least two image acquisition devices are different, and the at least two image acquisition devices correspond to the at least two frames of to-be-recognized images one by one, and the pedestrian is any pedestrian entering the target place;

[0011] A processing unit, configured to determine the identity information of the pedestrian according to the at least two frames of images to be recognized; and determine the passenger flow of the target venue according to the identity information of the pedestrian.

[0012] In a third aspect, an embodiment of the present application provides an electronic device, including: a processor, the processor is connected to a memory, the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the electronic device executes the method described in the first aspect.

[0013] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and the computer program enables a computer to execute the method described in the first aspect.

[0014] In a fifth aspect, an embodiment of the present application provides a computer program product, where the computer program product includes a non-transitory computer-readable storage medium storing a computer program, and the computer is operable to enable a computer to execute the method described in the first aspect.

[0015] Implementing the embodiments of the present application has the following beneficial effects:

[0016] It can be seen that in the embodiments of the present application, at least two frames of images to be recognized are collected by at least two image acquisition devices. Since the shooting angles of the at least two image acquisition devices are different, the at least two frames of images can include images to be recognized of the pedestrian from various angles. The identity of the pedestrian is recognized from various angles. In this way, even if some angles are blocked and the captured image information is not rich enough, the identity information of the pedestrian can be recognized by using the relatively rich images to be recognized from other angles, improving the comprehensiveness and accuracy of the identity information recognition, and further improving the statistical accuracy of the passenger flow. Description of the Drawings

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0018] Figure 1 It is a schematic flowchart of a passenger flow statistics system provided by an embodiment of the present application;

[0019] Figure 2 It is a schematic flowchart of a passenger flow statistics method provided by an embodiment of the present application;

[0020] Figure 3A schematic diagram for splicing human features provided by an embodiment of the present application;

[0021] Figure 4 A schematic flowchart of another passenger flow statistics method provided by an embodiment of the present application;

[0022] Figure 5 A block diagram of the functional units of a passenger flow statistics device provided by an embodiment of the present application;

[0023] Figure 6 A schematic structural diagram of a passenger flow statistics device provided by an embodiment of the present application. Detailed implementation manners

[0024] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without making creative efforts fall within the protection scope of the present application.

[0025] The terms "first", "second", "third", "fourth", etc. in the specification and claims of the present application and the accompanying drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but optionally further includes steps or units not listed, or optionally further includes other steps or units inherent to these processes, methods, products, or devices.

[0026] Referring to "embodiment" in this article means that a specific feature, result, or characteristic described in conjunction with the embodiment can be included in at least one embodiment of the present application. The phrase appears in various positions in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art explicitly and implicitly understand that the embodiments described herein can be combined with other embodiments.

[0027] To facilitate the understanding of the embodiments of the present application, the application scenarios of the embodiments of the present application will be first explained.

[0028] The passenger flow of the target venue involved in the embodiments of the present application can be understood as the number of people entering the target venue within a day. The target venue can be a shopping mall, supermarket, tourist attraction, amusement park, park, railway station, and so on. In the present application, the target venue is taken as a shopping mall as an example for illustration. In addition, the image acquisition device involved in the present application can be an analog camera, digital camera, standard definition camera, high definition camera, CCD camera, COMS camera, or other devices with image capture functions. The present application is not limited thereto.

[0029] It should be understood that if two pedestrians enter the target venue at the same time, the identity of these two pedestrians needs to be recognized simultaneously to obtain the identity information of each pedestrian, and the passenger flow of the target venue is determined according to the identity information of each pedestrian. In the present application, the process of recognizing the identity information of one pedestrian is taken as an example for illustration. The process of recognizing the identity information of other pedestrians is similar to that of this pedestrian and will not be described again.

[0030] Refer to Figure 1 , Figure 1 FIG. is a schematic diagram of the architecture of a passenger flow statistics system provided by an embodiment of the present application. The passenger flow statistics system is applied to a target venue. The passenger flow statistics system includes a passenger flow statistics device 10 and at least two image acquisition devices 20. Among them, the passenger flow statistics device 10 is in communication connection with each image acquisition device 20, and each image acquisition device 20 is arranged at the entrance of the target venue, and the capture angles of at least two image acquisition devices 20 are different.

[0031] Exemplarily, each image acquisition device 20 acquires at least two frames of images to be recognized of any pedestrian entering from the entrance of the target venue; each image acquisition device 20 sends the images to be recognized it has acquired to the passenger flow statistics device 10. The passenger flow statistics device 10 determines the identity information of the pedestrian according to the at least two frames of images to be recognized, and determines the passenger flow of the target venue according to the identity information of the pedestrian. That is to say, when the identity information of the pedestrian is that of a staff member of the target venue, the pedestrian is not regarded as the passenger flow of the target venue; when the identity information of the pedestrian is that of a non-staff member of the target venue, the pedestrian is regarded as a passenger flow of the target venue.

[0032] It can be seen that in the embodiments of the present application, at least two frames of images to be recognized are acquired by at least two image acquisition devices 20. Since the shooting angles of at least two image acquisition devices are different, the at least two frames of images can include images to be recognized of pedestrians from various angles. Therefore, in the process of the passenger flow statistics device 10 determining the identity information of the pedestrian according to the at least two frames of images to be recognized, the accuracy of identity information recognition can be improved, and further the statistical accuracy of the passenger flow can be improved.

[0033] Refer toFigure 2 , Figure 2 is a schematic flowchart of a passenger flow statistics method provided by an embodiment of this application. This method is applied to a passenger flow statistics device. The method includes the following steps:

[0034] 201: The passenger flow statistics device acquires at least two frames of images to be recognized of pedestrians through at least two image acquisition devices, where the shooting angles of the at least two image acquisition devices are different, and the at least two image acquisition devices correspond one-to-one to the at least two frames of images to be recognized, and the pedestrian is any pedestrian entering the target venue.

[0035] Exemplarily, the at least two image acquisition devices are arranged at the entrance of the target venue and at different positions, so that the at least two image acquisition devices can capture the pedestrians entering the target venue from different angles to obtain at least two frames of images to be recognized, where each image acquisition device acquires one frame of image to be recognized.

[0036] 202: The passenger flow statistics device determines the identity information of the pedestrian according to the at least two frames of images to be recognized.

[0037] Exemplarily, when any one of the at least two frames of images to be recognized includes the face of the pedestrian, specifically, the pedestrian can be tracked to obtain the area of the pedestrian in each frame of image to be recognized, and face detection is performed within this area. If a face is detected, it is determined that the frame of image to be recognized contains the face of the pedestrian, and then face recognition is performed using this frame of image to be recognized to obtain the identity information of the pedestrian. That is to say, when a face is detected, face recognition is directly used for identity recognition. When each frame of the at least two frames of images does not include the face of the pedestrian, the identity information of the pedestrian can be obtained according to the body features of the pedestrian in each frame of image.

[0038] Exemplarily, object detection, that is, human body detection, is performed on each frame of image to be recognized to obtain the human body box corresponding to the pedestrian in each frame of image. For example, object detection of the human body can be performed through common object detection networks, such as the R-CNN network, the PRN network, and so on. Specifically, object detection can be performed on each frame of image to be recognized to obtain the candidate box of the object (human body), and then non-maximum suppression is performed on the candidate box of the human body to remove duplicate candidate boxes to obtain the human body box corresponding to the pedestrian; then, feature extraction is performed on the image framed by the human body box to obtain the body features of the pedestrian in each frame of image to be recognized, that is, the feature map corresponding to the human body of the pedestrian.

[0039] Then, the human body features (feature maps) of the pedestrian in each frame of the image to be recognized are spliced to obtain the target human body features (target feature maps) corresponding to the pedestrian. Exemplarily, the multiple frames of images to be recognized can be fused pairwise in sequence according to the numbers of the acquisition devices to obtain the target human body features. Specifically, in the process of fusing any two human body features, the overlapping part between the two human body features is determined, the average value of the overlapping part is taken, and the non-overlapping parts are spliced. As Figure 3 shown, the human bodies represented by the first 4 columns of feature Figure 1 and feature Figure 2 are the overlapping parts, and the fifth column in feature Figure 2 is the non-overlapping part. Therefore, the pixel values of the first four columns are averaged, and the fifth column of feature Figure 2 is spliced to obtain the target feature map. As Figure 3 shown, the average values of a11 and b11, and a14 and b14 are respectively used as a pixel value of the target feature, and the pixel values of the non-overlapping parts, such as b15, are directly spliced. In this way, the repeated human body features are averaged to make the human body features in the overlapping part more accurate, and the non-overlapping human body features are spliced. Thus, after the at least two human body features are completely spliced, a complete human body feature, that is, the target human body feature corresponding to the pedestrian, can be obtained. Identity recognition is performed through the complete human body feature, improving the accuracy of identity recognition.

[0040] Further, feature extraction is performed on the target human body features of the pedestrian to obtain the target feature vector of the pedestrian. For example, the target human body features of the pedestrian can be subjected to feature extraction through multiple convolutional layers to obtain the target feature vector of the pedestrian; then, the target feature vector of the pedestrian is matched with each template vector in the template library to obtain the matching value corresponding to each template vector; in the case where the maximum matching value among the matching values corresponding to each template vector is greater than the first threshold, it is determined that the identity of the pedestrian is a staff member of the target venue; in the case where the maximum matching value among the matching values corresponding to each template vector is less than or equal to the first threshold, it is determined that the pedestrian is a non-staff member of the workplace.

[0041] Exemplarily, after obtaining the human body features of the pedestrian in each frame of the image to be recognized, it is also possible not to splice the human body features corresponding to the multiple frames of the image to be recognized to improve the efficiency of identity recognition. Specifically, according to the human body features (feature maps) of the pedestrian in each frame of the image to be recognized, the feature vector corresponding to each frame of the image to be recognized is obtained; then, the feature vector corresponding to each frame of the image to be recognized is matched with each template vector in the template library to obtain the matching values corresponding to each template vector; finally, the matching values corresponding to each frame of the image to be recognized and each template vector are weighted to obtain the matching value between the pedestrian and each template vector. When the maximum matching value among the matching values between the pedestrian and each template vector is greater than the second threshold, it is determined that the identity of the pedestrian is a staff member of the target venue; when the maximum matching value is less than or equal to the second threshold, it is determined that the identity of the pedestrian is a non-staff member of the target venue.

[0042] Specifically, in the process of performing target detection on each frame of the image to be recognized, while obtaining the human body frame of the pedestrian, a mask map is also generated for each human body frame. The mask map is used to represent the pixel points belonging to the pedestrian's human body and the pixel points not belonging to the pedestrian's human body in the candidate box. For example, if the mask map is obtained through one-hot encoding, when the pixel value is 1, it means that the pixel point belongs to the human body, and when the pixel value is 0, it means that the pixel point does not belong to the human body; then, the mask area of the mask map corresponding to the human body frame in each frame of the image to be recognized is determined. The mask area corresponding to each frame of the image to be recognized reflects the area occupied by the pedestrian's human body in the human body frame of this frame of the image to be recognized; then, the mask area corresponding to each frame of the image to be recognized is normalized to obtain the weight coefficient corresponding to each frame of the image to be recognized; finally, according to the weight coefficient corresponding to each frame of the image to be recognized, the matching value between each frame of the image to be recognized and the template vector A is weighted to obtain the matching value between the pedestrian and the template vector A, where the template vector A is any one of the template vectors.

[0043] For example, the at least two images to be recognized include the image to be recognized 1, the image to be recognized 2, and the image to be recognized 3. If the mask areas of the image to be recognized 1, the image to be recognized 2, and the image to be recognized 3 are 18, 25, and 36 respectively, that is to say, there are 18, 25, and 36 pixel points belonging to the pedestrian's body in the image to be recognized 1, the image to be recognized 2, and the image to be recognized 3 respectively. After normalization, the weight coefficients of the image to be recognized 1, the image to be recognized 2, and the image to be recognized 3 are 18 / 79, 25 / 79, and 36 / 79 respectively. Moreover, the matching values between the feature vectors corresponding to the image to be recognized 1, the image to be recognized 2, and the image to be recognized 3 and the template vector A are 0.4, 0.35, and 0.6 respectively. Then, it is determined that the matching value between the pedestrian and the template vector A is 0.4 * 18 / 79 + 0.35 * 25 / 79 + 0.6 * 36 / 79 = 0.47.

[0044] It can be seen that in the embodiment of the present application, the weight coefficient is set for each image to be recognized according to the mask area of the human body in the human body frame in each image to be recognized. Since the mask area of the mask image of the human body reflects the proportion of the human body in the human body frame, that is, it reflects the confidence level of identity recognition through the human body features in this frame of the image to be recognized. Finally, the matching value is weighted according to the confidence level of each image to be recognized, so that the obtained matching values with each template vector are more accurate, and the accuracy of identity recognition is improved.

[0045] Among them, each template vector in the above template library is composed of the feature vectors corresponding to the human body features of the staff in the target venue. That is to say, the human body images of the staff in the target venue are pre-extracted for features to obtain the feature vectors corresponding to the human body features of each staff member. Then, the feature vector of each staff member is used as a template vector and stored in the template library, and thus each of the template vectors is obtained.

[0046] 203: The passenger flow statistics device determines the passenger flow of the target venue according to the identity information of the pedestrian.

[0047] Exemplarily, according to the identity information of the pedestrian, the number of non-staff members entering the target venue is determined, that is, the number of non-staff members entering the target venue within a day is determined, and the number of non-staff members is used as the passenger flow of the target venue.

[0048] It can be seen that in the embodiment of the present application, at least two images to be recognized are collected by at least two image acquisition devices. Since the shooting angles of the at least two image acquisition devices are different, the at least two images can include the images to be recognized of the pedestrian from various angles. Therefore, in the process of determining the identity information of the pedestrian according to the at least two images to be recognized, the accuracy of identity information recognition can be improved, and further the statistical accuracy of the passenger flow can be improved.

[0049] In an embodiment of the present application, after obtaining the feature vector corresponding to each frame of the image to be recognized according to the human body features of the pedestrian in each frame of the image to be recognized, the feature vectors corresponding to each frame of the image to be recognized may also be concatenated (horizontally) to obtain the target feature vector corresponding to the pedestrian; then, according to the target feature vector of the pedestrian, the identity information of the pedestrian is obtained. Among them, obtaining the identity information of the pedestrian according to the target feature vector of the pedestrian is similar to the implementation process in step 202 above and will not be described again.

[0050] In an embodiment of the present application, after obtaining the target feature vector of each pedestrian, in order to prevent double counting of a certain pedestrian and improve the statistical accuracy of passenger flow, the target feature vector of each pedestrian may also be temporarily stored as a template vector in the template library. For example, a part of the cache space may be preset in the template library to cache the temporarily stored target feature vectors; then, during the process of identity recognition, the target feature vector is also used as a template vector. Therefore, each template vector mentioned in the present application includes the feature vector of the staff and the target feature vector of the stored pedestrians. In this way, even if a pedestrian enters the target place again within a day, the pedestrian is regarded as a staff member of the target place, and the passenger flow will not be counted again, so as to ensure that the passenger flow of each pedestrian (who is not a staff member of the target place) is counted only once, and the problem of low statistical accuracy of passenger flow caused by the frequent entry and exit of pedestrians is solved.

[0051] Furthermore, a timer may be set to clear the target feature vectors temporarily stored in the template library when the timing arrives. For example, clear the target feature vectors temporarily stored in the cache space at the end of each day (for example, 23:59:59), thereby saving storage space.

[0052] Refer to Figure 4 , Figure 4 which is a schematic flowchart of another passenger flow statistics method provided by an embodiment of the present application. This method is applied to a passenger flow statistics device. The same content as that in the embodiment shown in Figure 2 is not described again here. The method of this embodiment includes the following steps:

[0053] 401: The passenger flow statistics device collects at least two frames of images to be recognized of pedestrians through at least two image acquisition devices, where the shooting angles of the at least two image acquisition devices are different, and the at least two image acquisition devices correspond one-to-one to the at least two frames of images to be recognized, and the pedestrian is any pedestrian entering the target place.

[0054] 402: The passenger flow statistical device extracts features from each frame of the to-be-recognized images among the at least two frames of images, and obtains a first feature map of each frame of the to-be-recognized images.

[0055] Exemplarily, the features of each frame of the to-be-recognized images can be extracted through a target detection network to obtain a first feature map of each frame of the to-be-recognized images. Among them, the target detection network can be a general image segmentation network, such as U-NET, V-NET, YOLO, etc.

[0056] 403: The passenger flow statistical device performs image segmentation on each frame of the to-be-recognized images according to the first feature map of each frame of the to-be-recognized images, and obtains a human body box corresponding to the pedestrian in each frame of the to-be-recognized images.

[0057] Exemplarily, the first feature map can be convolved through the convolutional layer in the target detection network to obtain the pixel points belonging to the human body in each frame of the to-be-recognized images; according to the pixel points belonging to the human body in each frame of the to-be-recognized images, a human body box corresponding to the pedestrian is obtained.

[0058] 404: The passenger flow statistical device performs a mapping process on the first feature map of each frame of the to-be-recognized images to obtain a second feature map corresponding to each frame of the to-be-recognized images, where the dimension of the second feature map corresponding to each frame of the to-be-recognized images is the same as the dimension of the human body box.

[0059] Exemplarily, bilinear interpolation processing can be used to perform a mapping process on the first feature map of each frame of the to-be-recognized images to obtain a second feature map of each frame of the to-be-recognized images. It should be understood that since the first feature map is obtained through feature extraction, the detailed information retained in the first feature map is the detailed information related to the human body in each frame of the to-be-recognized images, that is, some useless detailed information, such as the background, is filtered out.

[0060] 405: The passenger flow statistical device performs an attention mechanism process on the second feature map corresponding to each frame of the to-be-recognized images and the image selected in the human body box corresponding to the pedestrian in each frame of the to-be-recognized images, and obtains a third feature map corresponding to each frame of the to-be-recognized images.

[0061] Exemplarily, the second feature map corresponding to each frame of the to-be-recognized images can be multiplied point by point with the image (i.e., the pixel value matrix of the image) selected in the human body box corresponding to the pedestrian in each frame of the to-be-recognized images to obtain a third feature map corresponding to each frame of the to-be-recognized images. It should be understood that since the second feature map is mapped from the first feature map, after the second feature map is multiplied point by point with the pixel value matrix, the useless information in the human body box will be filtered out, and more human body information will be retained, thereby improving the accuracy of identity recognition using the third feature map.

[0062] 406: The passenger flow statistics device determines the identity information of the pedestrian according to the third feature map corresponding to each frame of the image to be recognized.

[0063] Exemplarily, as shown in step 202 above, the third feature maps corresponding to each frame of the image to be recognized can be first spliced to obtain a target feature map, and then, identity recognition can be performed; or, first recognize according to the third feature map corresponding to each frame of the image to be recognized, and finally perform weighted processing on the recognition results of each frame of the image to be recognized to obtain the identity information of the pedestrian. The specific recognition process can refer to the content shown in 202 and will not be described again.

[0064] 407: The passenger flow statistics device determines the passenger flow of the target place according to the identity information of the pedestrian.

[0065] It can be seen that in the embodiments of the present application, at least two frames of images to be recognized are collected by at least two image acquisition devices. Since the shooting angles of the at least two image acquisition devices are different, the at least two frames of images can include the images to be recognized of the pedestrian from various angles. Therefore, in the process of determining the identity information of the pedestrian according to the at least two frames of images to be recognized, the accuracy of the identity information recognition can be improved, and further the statistical accuracy of the passenger flow can be improved; in addition, in the process of performing identity recognition, an attention mechanism is also adopted, so that the extracted third feature map is a feature map related to the human body, thereby further improving the accuracy of identity recognition.

[0066] Refer to Figure 5 , Figure 5 The functional unit composition block diagram of a passenger flow statistics device provided by the embodiments of the present application. The passenger flow statistics device 500 includes: an acquisition unit 501 and a processing unit 502, where:

[0067] The acquisition unit 501 is configured to collect at least two frames of images to be recognized of a pedestrian through at least two image acquisition devices, where the shooting angles of the at least two image acquisition devices are different, and the at least two image acquisition devices correspond to the at least two frames of images to be recognized one by one, and the pedestrian is any pedestrian entering the target place;

[0068] The processing unit 502 is configured to determine the identity information of the pedestrian according to the at least two frames of images to be recognized; and determine the passenger flow of the target place according to the identity information of the pedestrian.

[0069] In some possible implementation manners, when the face of the pedestrian is not included in each frame of the image to be recognized, in terms of determining the identity information of the pedestrian according to the at least two frames of images to be recognized, the processing unit 502 is specifically configured to:

[0070] Feature extraction is performed on each of the at least two to-be-recognized images to obtain the human features of the pedestrian in each of the to-be-recognized images;

[0071] The human features of the pedestrian in each of the to-be-recognized images are spliced to obtain target human features;

[0072] According to the target human features, a target feature vector corresponding to the pedestrian is obtained;

[0073] The target feature vector is matched with each template vector in the template library to obtain a matching value corresponding to each template vector, where each template vector in the template library is composed of feature vectors corresponding to the human features of the staff in the target venue;

[0074] When the maximum matching value among the matching values corresponding to each template vector is greater than the first threshold, it is determined that the identity of the pedestrian is the staff in the target venue;

[0075] When the maximum matching value is less than or equal to the first threshold, it is determined that the identity of the pedestrian is a non-staff in the target venue.

[0076] In some possible implementation manners, when the face of the pedestrian is not included in each of the to-be-recognized images, in terms of determining the identity information of the pedestrian according to the at least two to-be-recognized images, the processing unit 502 is specifically configured to:

[0077] Feature extraction is performed on each of the at least two to-be-recognized images to obtain the human features of the pedestrian in each of the to-be-recognized images;

[0078] According to the human features of the pedestrian in each of the to-be-recognized images, a feature vector corresponding to each of the to-be-recognized images is obtained;

[0079] The feature vector corresponding to each of the to-be-recognized images is matched with each template vector in the template library to obtain a matching value corresponding to each template vector, where each template vector in the template library is composed of feature vectors corresponding to the human features of the staff in the target venue;

[0080] The matching values corresponding to each of the to-be-recognized images and each template vector are weighted to obtain the matching value between the pedestrian and each template vector;

[0081] When the maximum matching value among the matching values between the pedestrian and each template vector is greater than the second threshold, it is determined that the identity of the pedestrian is the staff in the target venue;

[0082] In the case that the maximum matching value is less than or equal to the second threshold, determine that the identity of the pedestrian is a non-staff member of the target venue.

[0083] In some possible implementation manners, when extracting features from each of the at least two to-be-recognized images to obtain the human body features of the pedestrian in each of the at least two to-be-recognized images, the processing unit 502 is specifically configured to:

[0084] Perform object detection on each of the at least two to-be-recognized images to obtain a human body box corresponding to the pedestrian in each image;

[0085] Extract features from the image framed by the candidate box of the pedestrian to obtain the human body features of the pedestrian in each of the at least two to-be-recognized images.

[0086] In some possible implementation manners, when performing weighted processing on the matching values corresponding to each of the at least two to-be-recognized images and the respective template vectors to obtain the matching values between the pedestrian and the respective template vectors, the processing unit 502 is specifically configured to:

[0087] Determine the mask area of the mask graph corresponding to the human body box in each of the at least two to-be-recognized images;

[0088] Perform normalization processing according to the mask area corresponding to each of the at least two to-be-recognized images to obtain the weight coefficient corresponding to each of the at least two to-be-recognized images;

[0089] According to the weight coefficient of each of the at least two to-be-recognized images, perform weighted processing on the matching value between each of the at least two to-be-recognized images and the template vector A to obtain the matching value between the pedestrian and the template vector A, where the template vector A is any one of the respective template vectors.

[0090] In some possible implementation manners, when determining the passenger flow of the target venue according to the identity information of the pedestrian, the processing unit 502 is specifically configured to:

[0091] Determine the number of non-staff members entering the target venue according to the identity information of the pedestrian;

[0092] Use the number of non-staff members entering the target venue as the passenger flow of the target venue.

[0093] Refer to Figure 6 , Figure 6 is a schematic structural diagram of an electronic device provided by an embodiment of the present application. As Figure 6As shown in the figure, the electronic device 600 includes a transceiver 601, a processor 602, and a memory 603. They are connected by a bus 604. The memory 603 is used to store computer programs and data, and can transmit the data stored in the memory 603 to the processor 602.

[0094] The processor 602 is used to read the computer program in the memory 603 and perform the following operations:

[0095] Control the transceiver 601 to collect at least two frames of images to be recognized of a pedestrian through at least two image acquisition devices, wherein the shooting angles of the at least two image acquisition devices are different, and the at least two image acquisition devices correspond to the at least two frames of images to be recognized one by one, and the pedestrian is any pedestrian entering the target venue;

[0096] Determine the identity information of the pedestrian according to the at least two frames of images to be recognized; determine the passenger flow of the target venue according to the identity information of the pedestrian.

[0097] In some possible implementation manners, when the face of the pedestrian is not included in each frame of the image to be recognized, in terms of determining the identity information of the pedestrian according to the at least two frames of images to be recognized, the processor 602 is specifically used to perform the following operations:

[0098] Extract features from each frame of the at least two frames of images to be recognized to obtain the human body features of the pedestrian in each frame of the image to be recognized;

[0099] Stitch the human body features of the pedestrian in each frame of the image to be recognized to obtain a target human body feature;

[0100] Obtain a target feature vector corresponding to the pedestrian according to the target human body feature;

[0101] Match the target feature vector with each template vector in the template library to obtain a matching value corresponding to each template vector, wherein each template vector in the template library is composed of feature vectors corresponding to the human body features of the staff in the target venue;

[0102] When the maximum matching value among the matching values corresponding to each template vector is greater than a first threshold, determine that the identity of the pedestrian is the staff in the target venue;

[0103] When the maximum matching value is less than or equal to the first threshold, determine that the identity of the pedestrian is a non-staff member in the target venue.

[0104] In some possible embodiments, when the face of the pedestrian is not included in each image to be recognized, in determining the identity information of the pedestrian based on the at least two images to be recognized, the processor 602 is specifically configured to perform the following operations:

[0105] Extract features from each of the at least two images to be recognized to obtain the human body features of the pedestrian in each image to be recognized;

[0106] Obtain the feature vector corresponding to each image to be recognized according to the human body features of the pedestrian in each image to be recognized;

[0107] Match the feature vector corresponding to each image to be recognized with each template vector in the template library to obtain the matching values corresponding to each template vector, where each template vector in the template library is composed of the feature vectors corresponding to the human body features of the staff in the target venue;

[0108] Perform weighted processing on the matching values corresponding to each image to be recognized and each template vector to obtain the matching value between the pedestrian and each template vector;

[0109] When the maximum matching value among the matching values corresponding to the pedestrian and each template vector is greater than the second threshold, determine that the identity of the pedestrian is the staff in the target venue;

[0110] When the maximum matching value is less than or equal to the second threshold, determine that the identity of the pedestrian is a non-staff member in the target venue.

[0111] In some possible embodiments, in extracting features from each of the at least two images to be recognized to obtain the human body features of the pedestrian in each image to be recognized, the processor 602 is specifically configured to perform the following operations:

[0112] Perform object detection on each of the at least two images to be recognized to obtain the human body box corresponding to the pedestrian in each image;

[0113] Extract features from the image framed by the candidate box of the pedestrian to obtain the human body features of the pedestrian in each image to be recognized.

[0114] In some possible embodiments, in performing weighted processing on the matching values corresponding to each image to be recognized and each template vector to obtain the matching value between the pedestrian and each template vector, the processor 602 is specifically configured to perform the following operations:

[0115] Determine the mask area of the mask map corresponding to the human body frame in each frame of the image to be recognized;

[0116] Perform normalization processing according to the mask area corresponding to each frame of the image to be recognized, and obtain the weight coefficient corresponding to each frame of the image to be recognized;

[0117] According to the weight coefficient of each frame of the image to be recognized, perform weighted processing on the matching value corresponding to each frame of the image to be recognized and the template vector A, and obtain the matching value between the pedestrian and the template vector A, where the template vector A is any one of the various template vectors.

[0118] In some possible implementation manners, in terms of determining the passenger flow of the target venue according to the identity information of the pedestrian, the processor 602 is specifically configured to perform the following operations:

[0119] Determine the number of non-staff members entering the target venue according to the identity information of the pedestrian;

[0120] Use the number of non-staff members entering the target venue as the passenger flow of the target venue.

[0121] Specifically, the above transceiver 601 may be Figure 5 The acquisition unit 501 of the passenger flow statistics device 500 in the above embodiment, and the above processor 602 may be Figure 5 The processing unit 502 of the passenger flow statistics device 500 in the above embodiment.

[0122] It should be understood that the passenger flow statistics device in the present application may include a smart phone (such as an Android phone, an iOS phone, a Windows Phone phone, etc.), a tablet computer, a palm computer, a notebook computer, a mobile Internet device MID (Mobile Internet Devices, abbreviated as: MID), or a wearable device, etc. The above passenger flow statistics device is only an example, not an exhaustive list, and includes but is not limited to the above passenger flow statistics device. In practical applications, the above passenger flow statistics device may further include: an intelligent vehicle terminal, a computer device, and so on.

[0123] The embodiment of the present application further provides a computer-readable storage medium, and the computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement some or all of the steps of any one of the passenger flow statistics methods recorded in the above method embodiments.

[0124] An embodiment of the present application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program. The computer program is operable to cause a computer to execute some or all of the steps of any one of the passenger flow statistical methods described in the above method embodiments.

[0125] It should be noted that, for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that the present application is not limited by the described action sequence, because according to the present application, some steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to the present application.

[0126] In the above embodiments, the descriptions of the respective embodiments have their own emphases. For the parts not detailed in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0127] In several embodiments provided by the present application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection can be through some interfaces. The indirect coupling or communication connection of the device or unit can be in an electrical or other form.

[0128] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place, or they can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0129] In addition, the functional units in each embodiment of the present application can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above integrated units can be implemented in the form of hardware or in the form of software program modules.

[0130] When the integrated unit is implemented in the form of a software program module and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present application. The aforementioned memory includes: various media such as USB flash drives, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), external hard drives, magnetic disks, or optical discs that can store program codes.

[0131] Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing relevant hardware through a program. This program can be stored in a computer-readable memory, and the memory can include: flash drives, read-only memories (English: Read-Only Memory, abbreviated: ROM), random access memories (English: Random Access Memory, abbreviated: RAM), magnetic disks, or optical discs, etc.

[0132] The above has introduced the embodiments of the present application in detail. Specific examples are used in this article to elaborate on the principle and implementation manner of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the present application; at the same time, for those of ordinary skill in the art, according to the idea of the present application, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present application.

Claims

1. A passenger flow statistics method, characterized in that, it includes: collecting at least two frames of images to be recognized of pedestrians through at least two image acquisition devices, wherein the shooting angles of the at least two image acquisition devices are different, and the at least two image acquisition devices correspond to the at least two frames of images to be recognized one by one, and the pedestrians are any pedestrians entering the target venue; determining the identity information of the pedestrians according to the at least two frames of images to be recognized; in the case that each frame of the images to be recognized does not include the face of the pedestrians, it includes: performing feature extraction on each frame of the at least two frames of images to be recognized to obtain the human body features of the pedestrians in each frame of the images to be recognized; splicing the human body features of the pedestrians in each frame of the images to be recognized to obtain target human body features; obtaining a target feature vector corresponding to the pedestrians according to the target human body features; matching the target feature vector with each template vector in the template library to obtain a matching value corresponding to each template vector, wherein each template vector in the template library is composed of feature vectors corresponding to the human body features of the staff in the target venue; in the case that the maximum matching value among the matching values corresponding to each template vector is greater than the first threshold, determining that the identity of the pedestrians is the staff in the target venue; in the case that the maximum matching value is less than or equal to the first threshold, determining that the identity of the pedestrians is a non-staff member in the target venue; or, performing feature extraction on each frame of the at least two frames of images to be recognized to obtain the human body features of the pedestrians in each frame of the images to be recognized; obtaining a feature vector corresponding to each frame of the images to be recognized according to the human body features of the pedestrians in each frame of the images to be recognized; matching the feature vector corresponding to each frame of the images to be recognized with each template vector in the template library to obtain a matching value corresponding to each template vector, wherein each template vector in the template library is composed of feature vectors corresponding to the human body features of the staff in the target venue; performing weighted processing on the matching values corresponding to each frame of the images to be recognized and each template vector to obtain a matching value between the pedestrians and each template vector; in the case that the maximum matching value among the matching values between the pedestrians and each template vector is greater than the second threshold, determining that the identity of the pedestrians is the staff in the target venue; in the case that the maximum matching value is less than or equal to the second threshold, determining that the identity of the pedestrians is a non-staff member in the target venue; determining the passenger flow of the target venue according to the identity information of the pedestrians.

2. The method according to claim 1, characterized in that, the performing feature extraction on each frame of the at least two frames of images to be recognized to obtain the human body features of the pedestrians in each frame of the images to be recognized includes: performing target detection on each frame of the at least two frames of images to be recognized to obtain a human body frame corresponding to the pedestrians in each frame of the images; Extract features from the image framed by the candidate box of the pedestrian to obtain the human features of the pedestrian in each frame of the image to be recognized.

3. The method according to claim 2, wherein, the step of performing weighted processing on the matching values corresponding to the respective template vectors for each frame of the image to be recognized to obtain the matching values between the pedestrian and the respective template vectors includes: Determine the mask area of the mask map corresponding to the human body box in each frame of the image to be recognized; Perform normalization processing according to the mask area corresponding to each frame of the image to be recognized to obtain the weight coefficient corresponding to each frame of the image to be recognized; According to the weight coefficient of each frame of the image to be recognized, perform weighted processing on the matching value corresponding to the template vector A for each frame of the image to be recognized to obtain the matching value between the pedestrian and the template vector A, where the template vector A is any one of the respective template vectors.

4. The method according to claim 3, wherein, the step of determining the passenger flow of the target place according to the identity information of the pedestrian includes: Determine the number of non-staff members entering the target place according to the identity information of the pedestrian; Use the number of non-staff members entering the target place as the passenger flow of the target place.

5. A passenger flow statistics device, wherein, it includes: An acquisition unit for acquiring at least two frames of images to be recognized of a pedestrian through at least two image acquisition devices, wherein the shooting angles of the at least two image acquisition devices are different, and the at least two image acquisition devices correspond one-to-one to the at least two frames of images to be recognized, and the pedestrian is any pedestrian entering the target place; A processing unit for determining the identity information of the pedestrian according to the at least two frames of images to be recognized; in the case where the face of the pedestrian is not included in each frame of the image to be recognized, it includes: Extract features from each frame of the at least two frames of images to be recognized to obtain the human features of the pedestrian in each frame of the image to be recognized; splice the human features of the pedestrian in each frame of the image to be recognized to obtain a target human feature; obtain a target feature vector corresponding to the pedestrian according to the target human feature; match the target feature vector with each template vector in the template library to obtain matching values corresponding to the respective template vectors, where the respective template vectors in the template library are composed of feature vectors corresponding to the human features of the staff members of the target place; in the case where the maximum matching value among the matching values corresponding to the respective template vectors is greater than a first threshold, determine that the identity of the pedestrian is a staff member of the target place; in the case where the maximum matching value is less than or equal to the first threshold, determine that the identity of the pedestrian is a non-staff member of the target place; Or, Feature extraction is performed on each of the at least two to-be-recognized images to obtain the human features of the pedestrian in each to-be-recognized image; according to the human features of the pedestrian in each to-be-recognized image, a feature vector corresponding to each to-be-recognized image is obtained; the feature vector corresponding to each to-be-recognized image is matched with each template vector in the template library to obtain a matching value corresponding to each template vector, wherein each template vector in the template library is composed of the feature vectors corresponding to the human features of the staff in the target venue; the matching values corresponding to each to-be-recognized image and each template vector are weighted to obtain the matching value between the pedestrian and each template vector; when the maximum matching value among the matching values corresponding to the pedestrian and each template vector is greater than the second threshold, it is determined that the identity of the pedestrian is the staff in the target venue; when the maximum matching value is less than or equal to the second threshold, it is determined that the identity of the pedestrian is a non-staff in the target venue; According to the identity information of the pedestrian, the passenger flow of the target venue is determined.

6. An electronic device, characterized in that, it includes: a processor and a memory, the processor is connected to the memory, the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory so that the electronic device executes the method according to any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, the computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the method according to any one of claims 1-4.

Images