Passenger flow information statistics method, device, electronic equipment and storage medium

Abstract

The present invention discloses a passenger flow information statistics method, device, electronic device and storage medium, which are applicable to the field of computer vision technology. The method includes: obtaining an image to be processed; analyzing and processing the image to be processed to determine the punch-in identification information corresponding to the target passenger in the image to be processed; obtaining the real punch-in information corresponding to each passenger, the real punch-in information includes the real punch-in time and identification information corresponding to each passenger; matching the punch-in identification information with the real punch-in information, and determining the identification information of the target passenger from the real punch-in information; according to the identification information of the target passenger, associating the punch-in identification information with the real punch-in information, so as to count the passenger flow information corresponding to each passenger. The method is used to determine the punch-in time and punch-in place of each passenger, thereby realizing the statistics of passenger flow information. It is convenient for bus companies to dispatch buses according to passenger flow statistics information, thereby providing convenience for people's travel.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a passenger flow information statistics method, device, electronic equipment and storage medium. Background Art

[0002] Urban public transportation is a random service system that changes with passenger flow, road conditions, climate, etc. If the passenger flow survey information is not effective or the feedback is not timely or accurate, the dispatcher will not be able to conduct effective dispatching and command. Therefore, passenger flow information is the basis of the entire bus enterprise management business, and information technology can provide support and guarantee for the acquisition and processing of bus information. Comprehensive and accurate grasp of bus passenger flow is the basis of bus management work. It not only provides a basis for daily dispatch, but also provides important reference data for line network optimization.

[0003] At present, the existing technology can only obtain the data of passengers punching in on the time card puncher, but cannot link the data of passengers punching in on the time card puncher with the passengers themselves. Therefore, it is impossible to make reasonable statistics on bus passenger flow information. Summary of the invention

[0004] In view of this, the embodiments of the present invention provide a passenger flow information statistics method, device, electronic device and storage medium, aiming to solve the problem that reasonable statistics of bus passenger flow information cannot be performed.

[0005] According to a first aspect, an embodiment of the present invention provides a passenger flow information statistics method, the method comprising:

[0006] Get the image to be processed;

[0007] Analyze and process the image to be processed to determine the card identification information corresponding to the target passenger in the image to be processed;

[0008] Obtain the real clock-in information corresponding to each passenger, including the real clock-in time and identification information corresponding to each passenger;

[0009] Matching the punch card identification information with the real punch card information, and determining the identification information of the target passenger from the real punch card information;

[0010] According to the identification information of the target passenger, the punch-in identification information is associated with the real punch-in information so as to collect statistics on the passenger flow information corresponding to each passenger.

[0011] The passenger flow information statistics method provided by the embodiment of the present invention obtains the image to be processed, analyzes and processes the image to be processed, and determines the punch-in identification information corresponding to the target passenger in the image to be processed. Thus, the accuracy of the punch-in identification information corresponding to the determined target passenger can be guaranteed. In addition, the punch-in identification information of the target passenger is matched with the real punch-in information corresponding to each passenger obtained, and the identification information of the target passenger is determined from the real punch-in information. Thus, the accuracy of the identification information of the determined target passenger can be guaranteed. Then, according to the identification information of the target passenger, the punch-in identification information is associated with the real punch-in information, so as to count the passenger flow information corresponding to each passenger. In the above method, the electronic device associates the punch-in identification information with the real punch-in information, so as to determine the punch-in time and punch-in place of each passenger, thereby realizing the statistics of passenger flow information. It is convenient for bus companies to dispatch buses according to passenger flow statistics information, thereby providing convenience for people's travel.

[0012] According to the second aspect, an embodiment of the present invention further provides a passenger flow information statistics device, the device comprising:

[0013] A first acquisition module, used for acquiring an image to be processed;

[0014] A determination module is used to analyze and process the image to be processed, and determine the card identification information corresponding to the target passenger in the image to be processed;

[0015] The second acquisition module is used to obtain the real punch-in information corresponding to each passenger, and the real punch-in information includes the real punch-in time and identification information corresponding to each passenger;

[0016] A matching module, used to match the punch card identification information with the real punch card information, and determine the identification information of the target passenger from the real punch card information;

[0017] The statistics module is used to associate the punch-in identification information with the real punch-in information according to the identification information of the target passenger, so as to collect statistics on the passenger flow information corresponding to each passenger.

[0018] The passenger flow information statistics device provided by the embodiment of the present invention expands the identification punch-in time according to a preset ratio to obtain the identification punch-in time period. Since the electronic device may obtain multiple images to be processed of the target passenger punching in, the identification punch-in time determined according to the timestamp corresponding to one of the images to be processed may be different from the real punch-in time. Therefore, the electronic device expands the identification punch-in time according to a preset ratio to obtain the identification punch-in time period, ensuring that the identification punch-in time period can include the real punch-in time. Then the electronic device calculates the distance matrix between the real punch-in time corresponding to each passenger and the identification punch-in time period; analyzes the distance matrix, and outputs the matching results of each real punch-in time and the identification punch-in time period. Thereby, the accuracy of the matching results of each real punch-in time and the identification punch-in time period output can be guaranteed, thereby ensuring the accuracy of the real punch-in time corresponding to the determined target passenger.

[0019] According to the third aspect, an embodiment of the present invention provides an electronic device, including a memory and a processor, the memory and the processor are communicatively connected to each other, computer instructions are stored in the memory, and the processor executes the passenger flow information statistics method in the first aspect or any one of the embodiments of the first aspect by executing the computer instructions.

[0020] According to the fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable a computer to execute the passenger flow information statistics method in the first aspect or any one of the embodiments of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0022] Figure 1 is a flow chart of a passenger flow information statistics method provided by an embodiment of the present invention;

[0023] Figure 2 is a flow chart of a passenger flow information statistics method provided by another embodiment of the present invention;

[0024] Figure 3 is a flow chart of a passenger flow information statistics method provided by another embodiment of the present invention;

[0025] Figure 4 is a flow chart of a passenger flow information statistics method provided by another embodiment of the present invention;

[0026] Figure 5 is a flow chart of a passenger flow information statistics method provided by another embodiment of the present invention;

[0027] Figure 6 It is a functional module diagram of a passenger flow information statistics device provided by an embodiment of the present invention;

[0028] Figure 7 It is a schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0030] It should be noted that the method for passenger flow information statistics provided in the embodiment of the present application may be executed by a device for passenger flow information statistics, which may be implemented as part or all of an electronic device through software, hardware, or a combination of software and hardware, wherein the electronic device may be a server or a terminal, wherein the server in the embodiment of the present application may be a single server or a server cluster composed of multiple servers, and the terminal in the embodiment of the present application may be a smart phone, a personal computer, a tablet computer, a wearable device, an intelligent robot, or other intelligent hardware devices. In the following method embodiments, the execution subject is an electronic device as an example for explanation.

[0031] In one embodiment of the present application, Figure 1 As shown, a passenger flow information statistics method is provided, which is described by taking the application of the method to an electronic device as an example, and includes the following steps:

[0032] S11, obtaining an image to be processed.

[0033] In an optional implementation, the electronic device may acquire the image to be processed sent by the image acquisition device based on the connection between the electronic device and the image acquisition device.

[0034] In another optional implementation, the electronic device may also use its own image acquisition device to acquire the image to be processed.

[0035] In another optional implementation, the electronic device may receive the image to be processed sent by the other device based on the connection with the other device.

[0036] For example, the image acquisition device installed above the bus card reader can obtain the corresponding image to be processed during the passenger card-in process, and then the image acquisition device sends the corresponding image to be processed during the passenger card-in process to the electronic device based on the communication connection between the image acquisition device and the electronic device. The electronic device receives the image to be processed sent by the image acquisition device.

[0037] S12: Analyze and process the image to be processed to determine the card punching identification information corresponding to the target passenger in the image to be processed.

[0038] In an optional implementation, the electronic device can use a deep learning model to analyze and process the image to be processed, and output the punch-in identification information corresponding to the target passenger in the image to be processed. The punch-in identification information may include the punch-in identification time corresponding to the target passenger and whether the target passenger has punched in.

[0039] Among them, the deep learning model may include DNN (Deep Neural Networks), CNN (Convolutional Neural Networks), RNN (Recurrent Neural Network), etc. When the target artifact recognition model is CNN, it may be a V-Net model, a U-Net model, a Generative Adversarial Nets model, etc. The embodiment of the present application does not specifically limit the type of deep learning model.

[0040] Exemplarily, the electronic device can input the image to be processed into a pre-trained deep learning model, which performs feature extraction on the image to be processed and outputs the check-in recognition time corresponding to the target passenger in the image to be processed and the check-in information of the target passenger based on the extracted features.

[0041] This step will be described in detail below.

[0042] S13. Obtain the real check-in information corresponding to each passenger.

[0043] The real check-in information may include the check-in station, check-in time and identification information of each passenger.

[0044] In an optional implementation, the electronic device may obtain the real check-in information corresponding to each passenger sent by the check-in device based on the connection with the check-in device.

[0045] In another optional implementation, the electronic device may receive real check-in information corresponding to each passenger sent by other devices based on the connection with other devices.

[0046] Exemplarily, the real clock-in information corresponding to each passenger obtained by the electronic device can be expressed as: [...{ID(x):Time(x)}...], where ID(x) represents the identification information of the passenger in the real clock-in information, and Time(x) is the real clock-in time of the passenger in the real clock-in information.

[0047] S14. Match the punch-in identification information with the real punch-in information, and determine the identification information of the target passenger from the real punch-in information.

[0048] In an optional implementation, the electronic device may make a one-to-one correspondence between the punch-in identification information corresponding to the target passenger and the real punch-in information corresponding to each passenger, thereby determining the identification information of the target passenger from the real punch-in information corresponding to each passenger.

[0049] The identification information of the passenger may be the identity information of the passenger, such as the name of the passenger, the mobile phone number of the passenger, or the identification information of the passenger's punch-in object. For example, if the passenger punches in a bus card, the identification information of the passenger may be the number information of the public card. The embodiment of the present application does not specifically limit the identification information of the passenger.

[0050] Exemplarily, suppose the electronic device determines that the check-in time of the target passenger in the check-in identification information corresponding to the target passenger in the image to be processed is 10:00 by analyzing the image to be processed, and then the electronic device matches the check-in time of the target passenger in the check-in identification information of the target passenger with the check-in time of the target passenger in the check-in identification information, and determines that the passenger with a corresponding check-in time of 10:00 in the actual check-in information is the target passenger in the image to be processed, and determines that the identification of the target passenger is 122, then the electronic device determines that the identification information of the target passenger is 122 from the actual check-in information.

[0051] S15. According to the identification information of the target passenger, the punch-in identification information is associated with the real punch-in information so as to collect statistics on the passenger flow information corresponding to each passenger.

[0052] Specifically, the electronic device associates the clock-in identification information with the real clock-in information according to the identification information of the target passenger, and determines the real clock-in time of the target passenger and the boarding station of the target passenger.

[0053] Then, the electronic device identifies the features of the target passenger in the image to be processed, thereby obtaining feature information of the target passenger, wherein the feature information may be facial information of the target passenger or head, shoulder and neck information of the target passenger. The embodiment of the present application does not specifically limit the feature information.

[0054] The electronic device associates the characteristic information of the target passenger with the punch-in identification information and the real punch-in information, so that the electronic device can determine the target passenger's getting-off time and getting-off location from the getting-off record image according to the characteristic information of the target passenger.

[0055] According to the above content, the electronic device can obtain the boarding and alighting stations, boarding and alighting times of the target passengers on each bus line. Thus, the electronic device can obtain the boarding and alighting stations, boarding and alighting times of each passenger on each bus line, and the electronic device counts the boarding and alighting stations, boarding and alighting times of each passenger on each bus line to obtain passenger flow statistics information.

[0056] The passenger flow information statistics method provided by the embodiment of the present invention obtains the image to be processed, analyzes and processes the image to be processed, and determines the punch-in identification information corresponding to the target passenger in the image to be processed. Thus, the accuracy of the punch-in identification information corresponding to the determined target passenger can be guaranteed. In addition, the punch-in identification information of the target passenger is matched with the real punch-in information corresponding to each passenger obtained, and the identification information of the target passenger is determined from the real punch-in information. Thus, the accuracy of the identification information of the determined target passenger can be guaranteed. Then, according to the identification information of the target passenger, the punch-in identification information is associated with the real punch-in information, so as to count the passenger flow information corresponding to each passenger. In the above method, the electronic device associates the punch-in identification information with the real punch-in information, so as to determine the punch-in time and punch-in place of each passenger, thereby realizing the statistics of passenger flow information. It is convenient for bus companies to dispatch buses according to passenger flow statistics information, thereby providing convenience for people's travel.

[0057] In one embodiment of the present application, Figure 2 As shown, a passenger flow information statistics method is provided, which is described by taking the application of the method to an electronic device as an example, and includes the following steps:

[0058] S21, obtaining an image to be processed.

[0059] For details about this step, see the Figure 1 S11 is introduced in the previous section, and this step will not be repeated here.

[0060] S22: Analyze and process the image to be processed to determine the card punching identification information corresponding to the target passenger in the image to be processed.

[0061] For details about this step, see the Figure 1 S12 is introduced in the above, and this step will not be repeated here.

[0062] S23. Acquire the real clock-in information corresponding to each passenger, where the real clock-in information includes the real clock-in time and identification information corresponding to each passenger.

[0063] For details about this step, see the Figure 1 S13 is introduced in the above description, and this step will not be described in detail again.

[0064] S24. Match the punch-in identification information with the real punch-in information, and determine the identification information of the target passenger from the real punch-in information.

[0065] In an optional embodiment of the present application, the punch-in identification information includes the identification punch-in time corresponding to the target passenger. The identification punch-in time can be the time when the target passenger in the image to be processed completes the punch-in, and the timestamp of the image to be processed is extracted and used as the identification punch-in time.

[0066] In an optional embodiment, the electronic device can receive multiple consecutive frames of images to be processed sent by the image acquisition device, and then the electronic device identifies the passenger punching action in the multiple consecutive frames of images to be processed, and screens out the images to be processed with the passenger punching action.

[0067] Exemplarily, the electronic device identifies a list of passenger punch-in actions in a plurality of consecutive frames of images to be processed, which is represented as [...{Person(y):Time(y)}...], where Person(y) represents the passenger image index, and Time(y) is the corresponding identified punch-in time.

[0068] The above step S24 may further include the following steps:

[0069] S241. Match the identification clocking-in time corresponding to the target passenger with the actual clocking-in time corresponding to each passenger.

[0070] In an optional implementation of the present application, the electronic device may compare the identified clock-in time corresponding to the target passenger with the actual clock-in time corresponding to each passenger one by one to obtain a comparison result.

[0071] In an optional implementation of the present application, the above step S241 may further include the following steps:

[0072] (1) Expand the identification punch-in time according to a preset ratio to obtain the identification punch-in time period.

[0073] Specifically, since the clocking-in action is a continuous process, the same passenger may identify multiple clocking-in record time points, so the electronic device can expand the clocking-in time according to a preset ratio to obtain the clocking-in time period. The preset ratio can be to expand the clocking-in time forward by a preset time period, and then expand it backward by a preset time period.

[0074] Exemplarily, the identification punch-in time period can be expressed as follows:

[0075] p i =(pl i ,pr i )

[0076] Among them, pl i ,pr i Divided into left and right time points for identifying the punch-in time period, p i To identify the clock-in time.

[0077] (2) Calculate the distance matrix between the actual check-in time and the identified check-in time period corresponding to each passenger.

[0078] In an optional implementation, the electronic device may define a distance calculation function between the actual punch-in time and the identified punch-in time period:

[0079]

[0080] Then, the electronic device uses the distance calculation function to calculate the distance between the actual punch-in time and the identified punch-in time period corresponding to each passenger, and obtains a distance matrix.

[0081] In another optional implementation, the electronic device may compare the actual clock-in time corresponding to each passenger with the identified clock-in time period to determine a distance matrix between the actual clock-in time corresponding to each passenger and the identified clock-in time period.

[0082] (3) Analyze the distance matrix and output the matching results between the real punch-in time and the identified punch-in time period.

[0083] In an optional embodiment, the electronic device can compare the values ​​in each row in the distance matrix, and then select the first two digits with smaller values ​​from each row, and obtain the matching results of each real punch-in time and the identified punch-in time period based on the first two digits with smaller values ​​selected from each row.

[0084] Each row in the distance matrix may represent the distance between each real punch-in time and each identified punch-in time period.

[0085] Exemplarily, if there are 3 real punch-in times, there are 3 corresponding identification punch-in time periods. Then the first row of the distance matrix can represent the distance between the first real punch-in time and the three identification punch-in time periods, thus obtaining a 3×3 distance matrix. Then select the first two digits with smaller values ​​from the first row to represent the identification punch-in time period that the first real punch-in time may correspond to; then select the first two digits with smaller values ​​from the second row to represent the identification punch-in time period that the second real punch-in time may correspond to. Then select the first two digits with smaller values ​​from the third row to represent the identification punch-in time period that the third real punch-in time may correspond to. Then, based on the two values ​​selected in each row, perform permutations and combinations to obtain the matching results of each real punch-in time and the identification punch-in time period.

[0086] In another optional implementation, the electronic device may input the distance matrix into a bipartite graph maximum weight algorithm, and output a matching result between each actual clock-in time and the identified clock-in time period.

[0087] The bipartite graph maximum weight algorithm may be the Hungarian algorithm, or other algorithms that can match the real punch-in time with the identified punch-in time period to obtain a matching result.

[0088] S242. Determine the best matching result between the identified clock-in time and each real clock-in time according to the matching result, so as to determine the real clock-in time corresponding to the target passenger.

[0089] In an optional implementation, after the electronic device obtains the matching results of each real punch-in time and the identification punch-in time period, it analyzes the matching results and excludes the matching results of one identification time period corresponding to at least two real punch-in times and the matching results of at least two identification time periods corresponding to one real punch-in time. Then, from the remaining matching results, the matching result with the smallest sum of distances between each identification punch-in time period and each real punch-in time pair is determined, thereby determining the real punch-in time corresponding to the target passenger.

[0090] In an optional embodiment, the electronic device can input the distance matrix into a bipartite graph maximum weight algorithm. The bipartite graph maximum weight algorithm selects edges through a greedy algorithm based on the actual punch-in time and the identified punch-in time period to form a new bipartite graph. The maximum match is sought based on the bipartite graph, and when the maximum match is a complete match, the match can be determined to be the best match, thereby determining the best match result between the identified punch-in time and each actual punch-in time to determine the actual punch-in time corresponding to the target passenger.

[0091] S243. Determine the identification information of the target passenger according to the actual clocking-in time corresponding to the target passenger.

[0092] Specifically, the electronic device uses the real clock-in time corresponding to the target passenger and the real clock-in information corresponding to each passenger to determine the identification information of the target passenger.

[0093] S25. According to the identification information of the target passenger, the punch-in identification information is associated with the real punch-in information so as to collect statistics on the passenger flow information corresponding to each passenger.

[0094] For details about this step, see the Figure 1 S15 is introduced in the description, and this step will not be repeated here.

[0095] The passenger flow information statistics method provided by the embodiment of the present invention matches the identification punch-in time corresponding to the target passenger with the real punch-in time corresponding to each passenger, and determines the best matching result between the identification punch-in time and each real punch-in time according to the matching result, thereby determining the real punch-in time corresponding to the target passenger. The accuracy of the real punch-in time corresponding to the determined target passenger is guaranteed. Then, according to the real punch-in time corresponding to the target passenger, the identification information of the target passenger is determined, thereby ensuring the accuracy of the identification information of the determined target passenger. Instead of directly determining the punch-in time of the target passenger according to the identification punch-in time corresponding to the target passenger, which leads to the error of the punch-in time of the determined target passenger, further leads to the error of the identification information of the target passenger, resulting in confusion of the punch-in information of passengers on the bus and the error of the passenger flow statistics information. Therefore, the above passenger flow information statistics method can ensure that the identification information of the target passenger corresponds to the punch-in time of the target passenger and the image to be processed corresponding to the target passenger, thereby ensuring the accuracy of the passenger flow statistics information.

[0096] In addition, the passenger flow information statistics method provided by the embodiment of the present invention expands the identification punch-in time according to a preset ratio to obtain the identification punch-in time period. Since the electronic device may obtain multiple images to be processed of the target passenger punching in, the identification punch-in time determined according to the timestamp corresponding to one of the images to be processed may be different from the real punch-in time. Therefore, the electronic device expands the identification punch-in time according to a preset ratio to obtain the identification punch-in time period, ensuring that the identification punch-in time period can include the real punch-in time. Then the electronic device calculates the distance matrix between the real punch-in time corresponding to each passenger and the identification punch-in time period; analyzes the distance matrix, and outputs the matching results of each real punch-in time and the identification punch-in time period. Thereby, the accuracy of the matching results of each real punch-in time and the identification punch-in time period output can be guaranteed, thereby ensuring the accuracy of the real punch-in time corresponding to the determined target passenger.

[0097] In one embodiment of the present application, Figure 3 As shown, a passenger flow information statistics method is provided, which is described by taking the application of the method to an electronic device as an example, and includes the following steps:

[0098] S31, obtaining an image to be processed.

[0099] For details about this step, see the Figure 2 S21 is introduced in the above, and this step will not be described in detail here.

[0100] S32: Analyze and process the image to be processed to determine the card punching identification information corresponding to the target passenger in the image to be processed.

[0101] In an optional implementation of the present application, the above step S32 may further include the following steps:

[0102] S321, performing target recognition on the image to be processed, and determining the target passenger, punching object, and punching machine in the image to be processed.

[0103] Specifically, the electronic device can use the target detection model to perform target recognition on the image to be processed, and determine the target passenger, check-in object, and check-in machine in the target image.

[0104] Among them, the target detection model can be a DPM (Deformable Parts Model), and the target detection model can also be a model based on a convolutional neural network, such as a YOLO (You Only Look Once) detector, an R-CNN (Region-based Convolutional Neural Networks) model, an SSD (Single Shot MultiBox) detector, and a Mask R-CNN (MaskRegion-based Convolutional Neural Networks) model, etc. The embodiment of the present application does not specifically limit the target detection model.

[0105] S322: Perform action recognition on the target passenger to determine the clocking-in action of the target passenger.

[0106] In an optional implementation, the electronic device may input the identified target passenger into a deep learning model, perform action recognition on the target passenger, and determine the check-in action of the target passenger.

[0107] Among them, the deep learning model may include DNN (Deep Neural Networks), CNN (Convolutional Neural Networks), RNN (Recurrent Neural Network), etc. When the target artifact recognition model is CNN, it may be a V-Net model, a U-Net model, a Generative Adversarial Nets model, etc. The embodiment of the present application does not specifically limit the type of deep learning model.

[0108] S323: Determine that the target passenger has punched in based on the punching action of the target passenger and the positional relationship between the punching object and the punching device.

[0109] Specifically, after determining the clocking-in action of the target passenger, the electronic device obtains the positional relationship between the clocking-in object of the target passenger and the clocking-in device, and determines that the target passenger has clocked in based on the positional relationship between the clocking-in object of the target passenger and the clocking-in device.

[0110] S33. Acquire the real punch-in information corresponding to each passenger, where the real punch-in information includes the real punch-in time and identification information corresponding to each passenger.

[0111] For details about this step, see the Figure 2 S23 is introduced in the above description, and this step will not be described in detail here.

[0112] S34: Match the punch-in identification information with the real punch-in information, and determine the identification information of the target passenger from the real punch-in information.

[0113] For details about this step, see the Figure 2 S24 is introduced in , and this step will not be described in detail again.

[0114] S35. According to the identification information of the target passenger, the punch-in identification information is associated with the real punch-in information so as to collect statistics on the passenger flow information corresponding to each passenger.

[0115] For details about this step, see the Figure 2 S25 is introduced in the previous section, and this step will not be repeated here.

[0116] The passenger flow information statistics method provided by the embodiment of the present invention performs target recognition on the image to be processed, determines the target passenger, punch-in object and punch-in device in the image to be processed, and can ensure the accuracy of the target passenger, punch-in object and punch-in device determined by the electronic device. In addition, the punch-in action of the target passenger is determined by performing action recognition on the target passenger. Then, based on the punch-in action of the target passenger and the positional relationship between the punch-in object and the punch-in device, it can be accurately determined that the target passenger has punched in. The above method can use multiple layers of constraints to determine that the target passenger has punched in, based on the punch-in action of the target passenger and the positional relationship between the punch-in object and the punch-in device, thereby avoiding recognition errors that result in errors in the punch-in identification information corresponding to the determined target passenger.

[0117] In an optional embodiment of the present application, Figure 4 The above step S322 of “performing action recognition on the target passenger and determining the punching action of the target passenger” may include the following steps:

[0118] S221. Identify key points of the target passenger to obtain key points of the target passenger.

[0119] In an optional implementation, the electronic device may identify key points corresponding to various parts of the target passenger, thereby acquiring the key points of the target passenger.

[0120] In another optional implementation, the electronic device may also input the target passenger into the passenger key point model, and then output the key points of the target passenger.

[0121] The training process of the passenger key point model includes:

[0122] Obtain a passenger image and annotate key points of the passenger image, where the annotated key point set can be {top of head, neck, left shoulder, left elbow joint, left wrist, left hand center, right shoulder, right elbow joint, right wrist, right hand center}, and each key point is represented by (x, y, v), where x and y are the coordinates of the key point, and v indicates whether the key point is visible.

[0123] Then, the passenger images with marked passenger key points are used to train the general human key point algorithm framework to obtain a trained passenger key point model.

[0124] Specifically, the electronic device inputs the passenger image of the passenger key points into the human key point algorithm framework, extracts features through the deep backbone network, and finally outputs a heatmap probability map. The probability map represents the probability that each position on the image belongs to the key point of the corresponding category, and this probability map has the annotation results. Then a loss function such as MSE loss is used to measure the difference between the model output result and the annotation. The optimization goal is to minimize this difference as much as possible, so as to obtain a trained passenger key point model.

[0125] Among them, the human key point algorithm framework is an algorithm framework for identifying the key points of the human body, wherein the key points may include eyes, nose, mouth, ears, palms, etc. It can be trained according to the needs of the user. In the embodiment of the present application, the human key point algorithm framework is trained to obtain the passenger key point model, so that the passenger key point model can identify the key points of the target passenger's head, neck, left shoulder, left elbow joint, left wrist, left hand center, right shoulder, right elbow joint, right wrist, right hand center, etc.

[0126] S222. Identify key points of the target passenger and determine that the action of the target passenger is a clocking-in action.

[0127] In an optional implementation, after acquiring the key points of the target passenger, the electronic device may determine that the action of the target passenger is a clocking-in action based on the position information of each key point and whether each key point is visible.

[0128] For example, it is assumed that the electronic device determines that the right arm of the target passenger is extended to the right based on the position information of the target passenger's right shoulder, right elbow joint, right wrist, and right hand center, thereby determining that the target passenger's action is a clocking-in action.

[0129] In another optional implementation, after acquiring the key points of the target passenger, the electronic device extracts the image features of the key point position area of ​​the target passenger, and then generates the target passenger action recognition features based on the image features of the key point position area of ​​the target passenger. Then, the electronic device inputs the target passenger action recognition features into the action judgment binary classification model, and the binary classification model recognizes the input action recognition features, and determines that the action of the target passenger is a punch-in action based on the recognition result.

[0130] Among them, the binary classification model can be a linear regression model or a support vector machine model. The embodiment of the present application does not specifically limit the binary classification model.

[0131] The binary classification model can be trained based on the labeled key point location region images of passengers. The labeling information of the key point location region images of passengers can be 1 and 0, where 1 can represent that the key point location region image of the passenger is a check-in action, and 0 can represent that the key point location region image of the passenger is not a check-in action.

[0132] Specifically, the electronic device can input the labeled key point position area image of the passenger into the binary classification network, the binary classification network extracts features from the key point position area image of the passenger, and then outputs the results based on the extracted features, and then the electronic device uses the loss function to calculate the loss value between the output result of the binary classification network and the labeled information, and then continuously trains the binary classification network based on the loss value until the loss value between the output result of the binary classification network and the labeled information becomes smaller and smaller and tends to be stable, thereby determining that a binary classification model is obtained.

[0133] The passenger flow information statistics method provided by the embodiment of the present invention can ensure the accuracy of the acquired key points of the target passenger by identifying the key points of the target passenger. Then, the key points of the target passenger are identified and the action of the target passenger is determined to be a punch-in action, which can ensure the accuracy of the punch-in action of the determined target passenger.

[0134] In an optional embodiment of the present application, Figure 5 The above step S323 of “determining that the target passenger has clocked in based on the clocking-in action of the target passenger and the positional relationship between the clocking-in object and the clocking-in device” may include the following steps:

[0135] S231. Determine the binding relationship between the target passenger and the punch-in object based on the positional relationship between the key points of the target passenger and the punch-in object.

[0136] In an optional embodiment of the present application, the above S221 may include the following steps:

[0137] (1) Determine the hand area of ​​the target passenger based on the arm key points and palm key points among the key points of the target passenger.

[0138] (2) Based on the positions of the punch-in object and the hand area, calculate the intersection-over-union ratio between the hand area and the punch-in object.

[0139] (3) If the intersection-and-combination ratio is greater than the preset intersection-and-combination ratio threshold, the target passenger is bound to the check-in object.

[0140] Specifically, the electronic device can obtain the position information of the arm key points and the palm key points among the key points of the target passenger, and then determine the hand area of ​​the target passenger based on the position information of the arm key points and the palm key points.

[0141] For example, the electronic device calculates the distance from the left (right) elbow to the corresponding left (right) hand center, set as L, and then generates a square area with a side length of L / r with the left (right) hand center coordinate as the center, as the left (right) hand area frame of the target passenger. (Wherein, r is a preset parameter, representing the ratio of the length of the elbow to the palm, which can be obtained by statistics.)

[0142] Then, the electronic device determines the location information of the punch-in object according to the object detection model identified from the image to be processed. The electronic device calculates the intersection-and-union ratio between the hand area and the punch-in object based on the location information of the punch-in object and the position of the hand area. If the intersection-and-union ratio between the hand area and the punch-in object is greater than the preset intersection-and-union ratio threshold, the target passenger is bound to the punch-in object.

[0143] In an optional implementation, there may be multiple check-in objects in the image to be processed, and the electronic device can use the target detection model to obtain a list of corresponding check-in object detection frames {a1, a2, ..., aN} in the image to be processed. The electronic device calculates the intersection-and-union ratio between the hand area and the check-in object based on the position of the detection frame corresponding to each check-in object and the position of the hand area. The electronic device obtains the check-in object with the largest intersection-and-union ratio with the hand area of ​​the target passenger and determines it as the target check-in object.

[0144] Then, the electronic device calculates the intersection-and-union ratio between the hand area and the target punching object according to the positions of the target punching object and the hand area of ​​the target passenger. If the intersection-and-union ratio between the hand area and the target punching object is greater than a preset intersection-and-union ratio threshold, the target passenger is bound to the target punching object.

[0145] S232: When the target passenger's action is a clock-in action, obtain the distance between the clock-in object and the clock-in device.

[0146] Specifically, when the target passenger's action is a clocking-in action, the electronic device can determine the position information of the clocking-in object and the clocking-in device identified from the image to be processed by the target detection model, and then calculate the distance between the clocking-in object and the clocking-in device based on the position information of the clocking-in object and the clocking-in device.

[0147] S233: If the distance between the punching object and the punching device is less than the preset distance threshold, it is determined that the target passenger has punched in.

[0148] Specifically, the electronic device compares the distance between the punching object and the punching machine with a preset distance threshold. If the distance between the punching object and the punching machine is less than the preset distance threshold, it is determined that the target passenger has a punching action and the punching object has entered the punching range, and it is determined that the target passenger has punched in. Otherwise, it is determined that the target passenger has not punched in.

[0149] The passenger flow information statistics method provided by the embodiment of the present invention determines the binding relationship between the target passenger and the punch-in object based on the positional relationship between the key points of the target passenger and the punch-in object, so that the punch-in object corresponding to the target passenger can be quickly determined, and it can be determined that the target passenger uses the punch-in object to punch in. Then, when the target passenger's action is a punch-in action, the distance between the punch-in object and the punch-in device is obtained; if the distance between the punch-in object and the punch-in device is less than a preset distance threshold, it is determined that the target passenger has punched in. In the above method, when the target passenger is bound to the punch-in object, and the target passenger's action is a punch-in action, it can be accurately determined that the target passenger has punched in based on the distance between the punch-in object and the punch-in device being less than the preset distance threshold. Through multi-layer conditional constraints, it is determined that the target passenger has punched in, thereby avoiding recognition errors, which would result in errors in the punch-in identification information corresponding to the determined target passenger.

[0150] In addition, the passenger flow information statistics method provided by the embodiment of the present invention determines the hand area of ​​the target passenger based on the arm key points and the palm key points among the key points of the target passenger, thereby ensuring the accuracy of the hand area determined by the electronic device. In addition, based on the position of the punch-in object and the hand area, the intersection-and-union ratio between the hand area and the punch-in object is calculated. If the intersection-and-union ratio is greater than the preset intersection-and-union ratio threshold, it means that the target passenger is holding the punch-in object, and the target passenger is bound to the punch-in object. Thus, the accuracy of the binding relationship between the determined target passenger and the punch-in object can be ensured. This avoids the situation where the punch-in objects of other passengers are bound to the target passenger.

[0151] It should be understood that although Figure 1-5 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1-5 At least part of the steps may include multiple steps or multiple stages. These steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed in turn or alternately with other steps or at least part of the steps or stages in other steps.

[0152] like Figure 6 As shown, this embodiment provides a passenger flow information statistics device, the device comprising:

[0153] A first acquisition module 41, used to acquire an image to be processed;

[0154] A determination module 42 is used to analyze and process the image to be processed, and determine the card punching identification information corresponding to the target passenger in the image to be processed;

[0155] The second acquisition module 43 is used to acquire the real punch-in information corresponding to each passenger, and the real punch-in information includes the real punch-in time and identification information corresponding to each passenger;

[0156] A matching module 44, used to match the punch-in identification information with the real punch-in information, and determine the identification information of the target passenger from the real punch-in information;

[0157] The statistics module 45 is used to associate the punch-in identification information with the real punch-in information according to the identification information of the target passenger, so as to collect statistics on the passenger flow information corresponding to each passenger.

[0158] In one embodiment of the present application, the punch-in identification information includes the identification punch-in time corresponding to the target passenger, and the above-mentioned matching module 44 is specifically used to match the identification punch-in time corresponding to the target passenger with the real punch-in time corresponding to each passenger; based on the matching result, determine the best matching result between the identification punch-in time and each real punch-in time to determine the real punch-in time corresponding to the target passenger; based on the real punch-in time corresponding to the target passenger, determine the identification information of the target passenger.

[0159] In one embodiment of the present application, the above-mentioned matching module 44 is specifically used to expand the identification punch-in time according to a preset ratio to obtain the identification punch-in time period; calculate the distance matrix between the real punch-in time corresponding to each passenger and the identification punch-in time period; analyze the distance matrix, and output the matching results of each real punch-in time and the identification punch-in time period.

[0160] In one embodiment of the present application, the above-mentioned determination module 42 is specifically used to perform target recognition on the image to be processed, and determine the target passenger, punching object and punching machine in the image to be processed;

[0161] The target passenger's action is recognized to determine the target passenger's punching action; based on the target passenger's punching action and the positional relationship between the punching object and the punching device, it is determined that the target passenger has punched in.

[0162] In one embodiment of the present application, the above-mentioned determination module 42 is specifically used to identify key points of the target passenger and obtain the key points of the target passenger; identify the key points of the target passenger and determine that the action of the target passenger is a punch-in action.

[0163] In one embodiment of the present application, the above-mentioned determination module 42 is specifically used to determine the binding relationship between the target passenger and the check-in object based on the positional relationship between the key points of the target passenger and the check-in object; when the action of the target passenger is a check-in action, the distance between the check-in object and the check-in machine is obtained; if the distance between the check-in object and the check-in machine is less than the preset distance threshold, it is determined that the target passenger has checked in.

[0164] In one embodiment of the present application, the above-mentioned determination module 42 is specifically used to determine the hand area of ​​the target passenger based on the arm key points and palm key points among the key points of the target passenger; calculate the intersection-and-union ratio between the hand area and the check-in object based on the positions of the check-in object and the hand area; if the intersection-and-union ratio is greater than the preset intersection-and-union ratio threshold, the target passenger is bound to the check-in object.

[0165] For the specific limitations and beneficial effects of the passenger flow information statistics device, please refer to the limitations of the passenger flow information statistics method above, which will not be repeated here. Each module in the above passenger flow information statistics device can be implemented in whole or in part by software, hardware and a combination thereof. The above modules can be embedded in or independent of the processor in the electronic device in the form of hardware, or can be stored in the memory of the electronic device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0166] The embodiment of the present invention also provides an electronic device having the above Figure 6 The passenger flow information statistics device shown.

[0167] like Figure 7 As shown, Figure 7 is a schematic diagram of the structure of an electronic device provided by an optional embodiment of the present invention, such as Figure 7 As shown, the electronic device may include: at least one processor 51, such as a CPU (Central Processing Unit), at least one communication interface 53, a memory 54, and at least one communication bus 52. The communication bus 52 is used to realize the connection and communication between these components. The communication interface 53 may include a display screen (Display) and a keyboard (Keyboard), and the optional communication interface 53 may also include a standard wired interface and a wireless interface. The memory 54 may be a high-speed RAM memory (Random Access Memory) or a non-volatile memory (non-volatile memory), such as at least one disk storage. The memory 54 may optionally be at least one storage device located away from the aforementioned processor 51. The processor 51 may be combined with Figure 6 In the described device, the memory 54 stores an application program, and the processor 51 calls the program code stored in the memory 54 to execute any of the above method steps.

[0168] The communication bus 52 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The communication bus 52 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0169] Among them, the memory 54 may include a volatile memory (English: volatile memory), such as a random access memory (English: random-access memory, abbreviated: RAM); the memory may also include a non-volatile memory (English: non-volatile memory), such as a flash memory (English: flash memory), a hard disk drive (English: hard disk drive, abbreviated: HDD) or a solid-state drive (English: solid-state drive, abbreviated: SSD); the memory 54 may also include a combination of the above types of memory.

[0170] The processor 51 may be a central processing unit (CPU), a network processor (NP), or a combination of a CPU and a NP.

[0171] The processor 51 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0172] Optionally, the memory 54 is also used to store program instructions. The processor 51 can call the program instructions to implement the present application Figures 1 to 5 The passenger flow information statistics method shown in the embodiment.

[0173] The embodiment of the present invention also provides a non-transitory computer storage medium, which stores computer executable instructions, and the computer executable instructions can execute the passenger flow information statistics method in any of the above method embodiments. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory (Flash Memory), a hard disk (HDD) or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above types of memory.

[0174] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A passenger flow information statistics method, characterized in that: The method comprises: Get the image to be processed; Analyze and process the image to be processed to determine the card punching identification information corresponding to the target passenger in the image to be processed; Acquire the real clock-in information corresponding to each passenger, wherein the real clock-in information includes the real clock-in time and identification information corresponding to each passenger; Matching the punch-in identification information with the real punch-in information, and determining the identification information of the target passenger from the real punch-in information; According to the identification information of the target passenger, the punch-in identification information is associated with the real punch-in information, so as to collect statistics on the passenger flow information corresponding to each of the passengers; The step of associating the punch-in identification information with the real punch-in information according to the identification information of the target passenger so as to collect statistics on the passenger flow information corresponding to each of the passengers includes: According to the identification information of the target passenger, the punch-in identification information is associated with the real punch-in information to determine the real punch-in time of the target passenger and the boarding station of the target passenger; Identifying the features of the target passenger in the image to be processed to obtain feature information of the target passenger; Associating the characteristic information of the target passenger with the punch-in identification information and the real punch-in information; According to the characteristic information of the target passenger, determine the target passenger's getting off time and getting off station from the getting off record image; The boarding stations and the alighting stations, as well as the boarding times and the alighting times of the target passengers on each bus line are counted to obtain passenger flow statistical information.

2. The method according to claim 1, characterized in that The punch-in identification information includes the identification punch-in time corresponding to the target passenger, and the punch-in identification information is matched with the real punch-in information to determine the identification information of the target passenger from the real punch-in information, including: Matching the identified clock-in time corresponding to the target passenger with the actual clock-in time corresponding to each of the passengers; According to the matching result, determining the best matching result between the identified punch-in time and each of the real punch-in times, so as to determine the real punch-in time corresponding to the target passenger; The identification information of the target passenger is determined according to the actual clock-in time corresponding to the target passenger.

3. The method according to claim 2, characterized in that The matching of the identified clock-in time corresponding to the target passenger with the actual clock-in time corresponding to each of the passengers includes: Expanding the clock-in recognition time according to a preset ratio to obtain a clock-in recognition time period; Calculate the distance matrix between the actual clock-in time corresponding to each of the passengers and the identified clock-in time period; The distance matrix is ​​analyzed to output the matching results of each of the real punch-in times and the identified punch-in time periods.

4. The method according to claim 1, characterized in that: The analyzing and processing the image to be processed to determine the card punching identification information corresponding to the target passenger in the image to be processed includes: Performing target recognition on the image to be processed to determine the target passenger, the card-punching object, and the card-punching device in the image to be processed; Performing action recognition on the target passenger to determine the clocking-in action of the target passenger; According to the punching action of the target passenger and the positional relationship between the punching object and the punching device, it is determined that the target passenger has punched in.

5. The method according to claim 4, characterized in that The step of performing action recognition on the target passenger to determine the clocking-in action of the target passenger includes: Performing key point recognition on the target passenger to obtain the key points of the target passenger; The key points of the target passenger are identified, and the action of the target passenger is determined to be a clocking-in action.

6. The method according to claim 5, characterized in that The step of determining that the target passenger has clocked in according to the clocking-in action of the target passenger and the positional relationship between the clocking-in object and the clocking-in device comprises: Determine a binding relationship between the target passenger and the punch-in object according to a positional relationship between the key point of the target passenger and the punch-in object; When the target passenger's action is a card-punching action, obtaining the distance between the card-punching object and the card-punching device; If the distance between the punched object and the puncher is less than a preset distance threshold, it is determined that the target passenger has punched the card.

7. The method according to claim 6, characterized in that The step of determining the binding relationship between the target passenger and the check-in object according to the key points of the target passenger includes: Determine the hand area of ​​the target passenger according to the arm key points and the palm key points among the key points of the target passenger; Calculate the intersection-and-union ratio between the hand area and the punch-in object according to the positions of the punch-in object and the hand area; If the intersection-and-joint ratio is greater than a preset intersection-and-joint ratio threshold, the target passenger is bound to the check-in object.

8. A passenger flow information statistics device, characterized in that: The device comprises: A first acquisition module, used for acquiring an image to be processed; A determination module, used to analyze and process the image to be processed, and determine the card punching identification information corresponding to the target passenger in the image to be processed; A second acquisition module is used to acquire the real punch-in information corresponding to each passenger, wherein the real punch-in information includes the real punch-in time and identification information corresponding to each passenger; A matching module, used to match the punch-in identification information with the real punch-in information, and determine the identification information of the target passenger from the real punch-in information; A statistics module is used to associate the punch-in identification information with the real punch-in information according to the identification information of the target passenger, so as to count the passenger flow information corresponding to each of the passengers; wherein, the method of associating the punch-in identification information with the real punch-in information according to the identification information of the target passenger, so as to count the passenger flow information corresponding to each of the passengers, comprises: associating the punch-in identification information with the real punch-in information according to the identification information of the target passenger, and determining the real punch-in time of the target passenger and the boarding station of the target passenger; identifying the features of the target passenger in the image to be processed, and acquiring the feature information of the target passenger; associating the feature information of the target passenger with the punch-in identification information and the real punch-in information; determining the target passenger's alighting time and alighting station from the alighting record image according to the feature information of the target passenger; and counting the boarding station and the alighting station as well as the boarding time and the alighting time of the target passenger on each bus line to obtain passenger flow statistics information.

9. An electronic device, characterized in that: It comprises a memory and a processor, wherein the memory stores computer instructions, and the processor executes the passenger flow information statistics method according to any one of claims 1 to 7 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the passenger flow information statistics method according to any one of claims 1 to 7.

Images