A passenger boarding and alighting matching and OD demand estimation method based on bus door video

Abstract

The present invention discloses a method for matching passengers getting on and off the bus and estimating OD demand based on bus door videos, comprising: S1, identifying and tracking bus passengers to obtain the getting on and off information of each passenger; S2, extracting passenger features, which include passenger appearance features and passenger getting on and off the bus features; S3, calculating passenger appearance feature distances, and obtaining the distribution of passenger appearance feature distances through the calculation of passenger appearance feature distances; S4, matching passengers getting on and off the bus, and then performing bipartite graph matching based on the passenger getting on and off matching; S5, allocating OD amounts of unsuccessfully matched passengers; and S6, estimating bus OD demand.

Description

Technical Field

[0001] The present invention relates to the technical field of OD demand estimation, and in particular to a method for matching passengers getting on and off a bus and estimating OD demand based on bus door videos. Background Art

[0002] Extracting bus OD (Out-of-Position) (OD) is crucial for urban transportation system planning. Public transportation passenger boarding and alighting (OD) information provides crucial data support for public transportation operations and regulation. It can be used to assess the impact of planning on operators and passengers, and is also used in overall public transportation network planning and scheduling. Traditionally, OD demand has been inferred based on station passenger flow and allocation algorithms, primarily using OD matrices for flow allocation. However, traditional allocation algorithms face technical bottlenecks in improving estimation accuracy.

[0003] A literature review of existing technologies revealed that passenger OD is mostly obtained manually or using simple signal processing equipment. Data acquisition primarily involves the following methods: automatic fare collection systems, automatic passenger counting systems, mobile phone positioning systems, automatic vehicle positioning data, Wi-Fi data, RFID data, and video data. Representative research results include "Public Transit Passenger Flow OD Analysis Method, Device, and Storage Medium" (CN202210270926.7) and "Public Transit OD Estimation and Allocation Method Based on Multi-Source Data Fusion" (CN202111345199.8).

[0004] In most cases, bus ID cards are only used for entry and do not record the information of bus passengers getting off the bus. Therefore, it is necessary to infer the passengers' getting off stations. Traditional methods mostly use inference to estimate OD demand. Traditional bus OD estimation methods mainly include:

[0005] (1) OD demand estimation based on station attraction coefficients calculated from land properties. Inferring OD based on station attraction coefficients calculated from land properties requires considering passenger transfers between bus and rail transit, the transfer capacity of each station, and its surrounding environment. Representative papers include "Destination Estimation for Bus Passengers Based on Data Fusion."

[0006] (2) Estimating OD demand at alighting stops based on boarding / disembarking data. Estimating OD demand based on boarding / disembarking data primarily analyzes bus travel characteristics based on data obtained from bus IC card information. Representative papers include "Alighting stop determination using two-step algorithms in bus transit systems."

[0007] (3) Estimate OD demand at alighting stops based on boarding card swipe data. The characteristics of bus travel need to be considered when allocating boarding and alighting passenger flows. A reverse model is then established to estimate OD demand, using the number of boarding and alighting passengers at bus stops as a constraint. Representative papers include “Estimating Bus Loads and OD Flows Using Location-Stamped Farebox and Wi-Fi Signal Data.”

[0008] There are a variety of means and methods for OD demand estimation, but there is a lack of methods for OD demand estimation based on matching. Summary of the Invention

[0009] In view of the shortcomings of the existing technology, the present invention aims to provide a method for matching passengers getting on and off the bus and estimating OD demand based on bus door video, comprising:

[0010] S1. Identify and track bus passengers and obtain the boarding and alighting information of each passenger;

[0011] S2. Extracting passenger features, including passenger appearance features and passenger boarding and alighting features;

[0012] S3. Calculate the passenger appearance feature distance. The distribution of the passenger appearance feature distance can be obtained by calculating the passenger appearance feature distance.

[0013] S4, matching passengers getting on and off the bus, and then performing bipartite graph matching;

[0014] S5. Allocate OD amount of unsuccessfully matched passengers;

[0015] S6. Estimating bus OD demand.

[0016] Preferably, in step S1, a bus video from which passenger OD needs to be extracted is input, and Yolov5 and DeepSORT are used to identify and track passengers to obtain the boarding and alighting information of each passenger.

[0017] Preferably, in step S2, the passenger boarding and alighting features are extracted through passenger identification and tracking. The passenger appearance feature extraction requires first selecting the image with the highest recognition confidence of the same passenger, and then removing the background noise of the image through Yolov7 instance segmentation, so as to retain only the color information of the passenger itself.

[0018] Preferably, the likelihood function of the passenger appearance feature distance in step S3 is obtained by fusing the likelihood functions of the color feature distance and the deep learning high-dimensional feature distance. The passenger appearance feature distance includes calculating the color feature distance and calculating the deep learning high-dimensional feature distance. The likelihood functions of the passenger color feature distance and the deep learning high-dimensional feature distance are obtained through the color feature distance and the deep learning high-dimensional feature distance.

[0019] Preferably, the bipartite graph in step S4 includes a set of boarding passengers and a set of disembarking passengers. After determining the matching range, the passenger boarding and disembarking matching is constructed as a bipartite graph matching problem, and the best matching is performed with the goal of maximizing the total matching probability.

[0020] Preferably, in step S5, passenger boarding and alighting pairs with a higher matching probability are selected as credible partial matching results, and the remaining unmatched passengers are subjected to OD inference through the allocation algorithm. Based on the OD of the successfully matched passengers, the boarding and alighting passenger flow at each station corresponding to the unsuccessfully matched passengers is calculated respectively, and the boarding and alighting passenger flow is used as the input flow of the allocation algorithm.

[0021] Preferably, in step S6, the public transportation OD demand estimation is to obtain the public transportation OD demand result of the second part by integrating the successfully matched passengers with the allocation algorithm, and thus obtain the final passenger OD demand result.

[0022] Compared with the prior art, the present invention has the following advantages: the color feature C of the passenger image and the deep learning high-dimensional feature F are taken into consideration to establish the optimal matching relationship between boarding passengers and alighting passengers, which is suitable for estimating passenger OD demand of buses equipped with door video surveillance, and uses bus videos to identify individual passengers to obtain partial passenger OD samples to reduce estimation errors; passenger identification and tracking are performed based on Yolov5 and DeepSORT, and instance segmentation of passenger images in bus videos is performed using Yolov7, which effectively improves the accuracy of OD demand estimation. The present invention combines the passenger matching algorithm with the OD allocation algorithm, not only using image processing technology to realize passenger identification and appearance feature extraction of a single door video, but also determining the optimal matching of boarding and alighting passengers in different door videos on this basis, integrating the OD allocation algorithm and the partial passenger individual OD samples obtained by the passenger matching algorithm, and improving the accuracy of bus OD demand estimation. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 A flow chart of passenger getting on and off matching and OD demand estimation of the method for passenger getting on and off matching and OD demand estimation based on bus door video according to the present invention;

[0024] Figure 2 A flowchart of passenger boarding and alighting state recognition of a method for matching passenger boarding and alighting with OD demand estimation based on bus door video according to the present invention;

[0025] Figure 3 This is a diagram showing the result of processing the appearance features of passengers in bus door videos according to the method for matching passengers getting on and off the bus and estimating OD demand based on bus door videos of the present invention;

[0026] Figure 4 A route diagram of the Fengpu Express Line according to the method for matching passengers getting on and off the bus and estimating OD demand based on bus door videos of the present invention;

[0027] Figure 5 This is a distance distribution diagram of passenger appearance features according to the passenger boarding and alighting matching and OD demand estimation method based on bus door video according to the present invention. DETAILED DESCRIPTION

[0028] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0029] Reference Figure 1-5 , a passenger boarding and alighting matching and OD demand estimation method based on bus door video, including:

[0030] Step 1: Bus passenger identification and tracking. Input the bus video from which the passenger OD needs to be extracted, use Yolov5 and DeepSORT to identify and track the passengers, and obtain the boarding and alighting information of each passenger.

[0031] Step 2: Passenger feature extraction. Passenger feature extraction includes passenger appearance features and passenger boarding and alighting features. Passenger appearance feature extraction requires first selecting the image with the highest recognition confidence of the same passenger, and then removing the background noise of the image through Yolov7 instance segmentation, so as to retain only the color information of the passenger itself; Passenger boarding and alighting features are extracted through passenger identification and tracking, where the passenger boarding and alighting status can be obtained by Figure 2 As shown.

[0032] Step 3: Calculate the passenger appearance feature distance. The distribution of passenger appearance feature distances can be obtained by calculating the passenger appearance feature distance. The likelihood function of the passenger appearance feature distance is obtained by fusing the likelihood functions of the color feature distance and the deep learning high-dimensional feature distance.

[0033] (1) Calculate the color feature distance. For a pair of boarding passenger i and disembarking passenger j, their color features are recorded as and The color feature distance d between the two c(i, j) can be expressed by Bhattacharyya distance:

[0034]

[0035] (2) Calculate the distance of deep learning high-dimensional features. The deep learning high-dimensional features of passenger i getting on the bus and passenger j getting off the bus are recorded as and The distance between the two is d F (i, j), calculated using Manhattan distance, can be expressed as:

[0036]

[0037] (3) Fusion of passenger appearance feature distance likelihood function.

[0038] Through (1) and (2), the likelihood function of passenger color feature distance and deep learning high-dimensional feature distance can be obtained. The likelihood function of passenger appearance feature distance is composed of color feature distance d c (i, j) and deep learning high-dimensional feature distance d F (i, j) is obtained by merging the likelihood functions of the two. As shown in formulas (3) and (4), P(d C |δ ij =1) represents the likelihood function of the color feature distance, P(d F |δ ij =1) represents the likelihood function of deep learning high-dimensional feature distance, θ C represents the weight of the color feature, θ F Represents the weight of high-dimensional features for deep learning.

[0039]

[0040]

[0041] (4) Calculation of passenger characteristic distance distribution.

[0042] The fused passenger appearance feature distance distribution likelihood function is obtained from (3) and (4). They are respectively the passenger correct matching likelihood function P(d ij |δ ij =1) and the passenger mismatch likelihood function P(d ij |δ ij =0). Prior probability function P(δ ij =1 is the historical OD probability of public transportation.

[0043] P(d ij )=P(d ij |δ ij =1)P(δ ij =1)+P(d ij |δij =0)|P(δ ij =0) (5)

[0044] P(δ ij =0) = 1-P(δ ij =1)0 (6)

[0045] According to the Bayesian method, the posterior probability is shown in formula (7). Formula (8) calculates the matching probability Pij of the boarding passenger i and the getting-off passenger j:

[0046]

[0047] P ij =P(δ ij =1|d ij ) (8)

[0048] Step 4: Passengers are matched with each other.

[0049] Passenger boarding and disembarking matching is divided into two steps. First, the matching range of passengers needs to be determined, and then bipartite graph matching is performed. The bipartite graph contains the set of boarding passengers U and the set of disembarking passengers D. k and v represent the passenger boarding station and the passenger disembarking station respectively. For example, D2 represents the set of passengers who disembarked at station 2. Assuming that a passenger i boards the bus at station k, the matching disembarking passenger j can only disembark after station k. The set of passengers that may match passenger i is recorded as j∈S(i); after determining the matching range, the passenger boarding and disembarking matching is constructed as a bipartite graph matching problem, and the best matching is performed with the goal of maximizing the total matching probability, as shown in formula (9). Formula (10) converts the decision variable δ ij Specified as a binary integer, it is 1 if i and j match, otherwise it is 0; Formula (11) ensures that any boarding passenger i can only match at most one getting-off passenger j; Formula (12) ensures that any getting-off passenger j can only match at most one boarding passenger i.

[0050]

[0051]

[0052]

[0053]

[0054] Step 5: Allocate OD amount to unsuccessfully matched passengers.

[0055] Passenger boarding and alighting pairs with a higher matching probability are selected as credible partial matching results, and the remaining unmatched passengers are subjected to OD inference through the allocation algorithm.

[0056] (1) Set δ as the passenger boarding and disembarking matching probability threshold, and select P ij The matching pairs of >δ are regarded as successful matching passengers, and the corresponding boarding and alighting station information is obtained as the bus OD demand result S of the first part. ij .

[0057] (2) Unsuccessful matching of OD quantity allocation of passengers.

[0058] Based on the OD of the successfully matched passengers, the passenger flow of each station corresponding to the unsuccessfully matched passengers is calculated, and these passenger flows are used as the input flow of the allocation algorithm. As shown in formula (13) and formula (14), the number of passengers T at bus station i input by the allocation algorithm is i Equal to the number of passengers K boarding the bus door video recognition i The difference between the number of passengers Si who successfully boarded the bus at the station; the number of passengers who got off at bus station j input by the allocation algorithm E j Equal to the number of passengers V who get off the bus through the video recognition j The number of passengers who successfully matched the stop W j The difference.

[0059] T i =K i -S i (13)

[0060] E j =V j -W j (14)

[0061]

[0062] Formula (15) is used to calculate the historical OD probability of public transportation. is the historical OD probability of getting on at station i and getting off at station j; Q ij is the number of passengers who boarded at station i and got off at station j. According to the properties of probability distribution, the above probability must be normalized using formula (16):

[0063]

[0064] Among them, A i is the total probability of getting on the bus from station i and getting off at the remaining stations; is the normalized historical OD probability.

[0065] Calculate the attraction coefficient of the site. Define F ij Generate attraction coefficient for bus stop i and bus stop j:

[0066]

[0067] Among them, R is the distance between bus stop i and bus stop j; L is the average distance of bus travel.

[0068] According to the nature of probability distribution, the site attraction coefficient must be normalized:

[0069]

[0070] Among them, Bi refers to the total coefficient of attraction from station i to the rest of the stations, X ij is the OD quantity of passengers who failed to match bus stop i with bus stop j.

[0071]

[0072]

[0073]

[0074]

[0075]

[0076]

[0077] Formula (19) to Formula (24) are iterative formulas for inferring bus OD. and The number of people getting on the bus at the i-th station and the number of people getting off at the j-th station after iteration are respectively. The OD distribution of bus passenger flow is corrected using the correction coefficient, and the corrected OD distribution is recorded as The loop iterates until the correction coefficient θ is calculated for the tth time j and δ j The product of is close to 1, and the iteration stops when the set accuracy requirement is met. ij To calculate the OD distribution of unsuccessfully matched passengers.

[0078] Step 6: Final bus OD demand estimation

[0079] Comprehensive P ij >δ successfully matched passengers S ij The second part of the bus OD demand result X is obtained by the allocation algorithm ij , the final passenger OD demand result can be obtained.

[0080] L ij =S ij +X ij (25)

[0081] The key to matching passenger boarding and alighting and OD demand estimation based on bus door videos lies in identifying, tracking, and matching individual passengers to improve OD accuracy. Existing technologies fall short in addressing this issue. This invention leverages bus videos to identify individual passengers, thereby obtaining partial OD samples and reducing estimation errors. Passenger identification and tracking are performed using Yolov5 and DeepSORT, and instance segmentation of passenger images from bus videos is performed using Yolov7, effectively improving the accuracy of OD demand estimation.

[0082] Example 1:

[0083] The route of Fengpu Express in Example 1 of the present invention is as follows Figure 2 The Shanghai Fengpu Express is a typical BRT line with concentrated passenger demand. Operating from Fengxian District, it radiates to Pudong, Minhang, Songjiang, and Jinshan. It is approximately 20.6 kilometers long and has 13 stops along the route. Conventional buses travel an average of 99,000 kilometers per day, transporting 97,000 passengers per day.

[0084] The method of the present invention is used to estimate the OD demand of the Fengpu Express from Shendu Highway Station to Nanqiao Bus Station from 9:15 to 10:01 on October 12, 2021. The specific process is briefly described as follows:

[0085] Step 1: Bus passenger identification and tracking. Yolov5 and Deepsort are combined to implement passenger identification and tracking for the Fengpu Express. The passenger boarding and alighting information for the front door stop 10 is obtained as shown in Table 1.

[0086] Step 2: Passenger feature extraction. Passenger boarding and alighting features can be obtained by passenger identification and tracking. Table 1 shows the passenger boarding and alighting feature extraction results for the front door station 10 as an example. Passenger appearance features are obtained by instance segmentation. Figure 3 The results of passenger appearance feature processing in bus door videos.

[0087] Table 1 Passenger boarding and alighting feature extraction

[0088]

[0089]

[0090] (2) Passenger appearance feature extraction

[0091] The passenger appearance extraction results are as follows: Figure 3 As shown in the figure, Yolov7 is used to perform image instance segmentation, which removes the noise of the original passenger image and obtains a passenger image with clear contours after segmentation.

[0092] Step 3: Calculate the passenger appearance feature distance.

[0093] The distance between the appearance features of the passenger images is calculated. Figure 5 The distance distribution of passenger appearance features. The likelihood function curve is derived from the training set, and correctly matched passengers are obtained by manual matching. The dashed line represents the probability distribution of distances between correctly matched pairs of passengers, while the solid line represents the probability distribution of distances between incorrectly matched pairs of passengers.

[0094] Step 4: Passengers are matched with each other.

[0095] Select P ij Matching pairs with a value greater than 0.39 are considered as successful matching passengers. ij When ≥0.39, more than 70% of the passenger re-identification results are matched correctly. The examples of successful matching results of some passengers shown in Table 2 and the successful matching results of some passengers shown in Table 3 are obtained as the first part of the bus OD demand result S ij .

[0096] Table 2 Examples of successful matching results for some passengers

[0097]

[0098] Table 3 Successful OD matching results for some passengers

[0099]

[0100] Step 5: Allocate the OD amount of unsuccessfully matched passengers. The calculation results of the OD amount of unsuccessfully matched passengers are shown in Table 4.

[0101] Table 4 Passenger flow at stations where passengers were not successfully matched

[0102]

[0103]

[0104] Through formulas (13)-(24), the OD distribution results of unsuccessfully matched passengers are obtained as shown in Table 5.

[0105] Table 5. OD results of unsuccessfully matched passengers

[0106]

[0107]

[0108] Step 6: Final bus OD demand estimation. ij >0.39 successfully matched passengers S ij The second part of the bus OD demand result X is obtained by the allocation algorithm ij, we obtain the final passenger OD demand results shown in Table 6. By comparing the errors of Method 1 (Cao and Wang's OD demand estimation method based only on station boarding passenger flow), Method 2 (Zhou Xuemei's method based only on station boarding and alighting passenger flow), and the method proposed in this paper, we obtain the RMSE errors of OD demand estimation for the three methods shown in Table 7.

[0109] Compared to the method based solely on passenger flow at a station (Method 1), the present invention (Method 3) reduces the RMSE error by approximately 84%; compared to the method based solely on passenger flow at a station (Method 2), the RMSE error is reduced by approximately 66%. Therefore, the present invention can effectively improve the estimation accuracy of bus OD.

[0110] Table 6 Final passenger OD demand results

[0111]

[0112]

[0113] Table 7 RMSE error of OD demand estimation using three methods

[0114]

[0115] The number of devices and processing scales described herein are intended to simplify the description of the present invention, and applications, modifications, and variations of the present invention will be apparent to those skilled in the art.

[0116] Although the embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the description and implementation methods. They can be fully applied to various fields suitable for the present invention. For those familiar with the art, additional modifications can be easily implemented. Therefore, without departing from the general concept defined by the claims and the scope of equivalents, the present invention is not limited to the specific details and illustrations shown and described herein.

Claims

1. A method for matching passengers getting on and off buses and estimating OD demand based on bus door videos, characterized by: The following steps are involved: S1. Identify and track bus passengers and obtain the boarding and alighting information of each passenger; S2. Extracting passenger features, including passenger appearance features and passenger boarding and alighting features; To extract passenger appearance features, we first select the image with the highest recognition confidence among the same passenger. Then, we use Yolov7 instance segmentation to remove background noise from the image, thus retaining only the passenger's color information. Passenger boarding and alighting features are extracted through passenger identification and tracking. S3. Calculate the passenger appearance feature distance. The distribution of the passenger appearance feature distance can be obtained by calculating the passenger appearance feature distance. The likelihood function of the passenger appearance feature distance is obtained by fusing the likelihood functions of the color feature distance and the deep learning high-dimensional feature distance. The passenger appearance feature distance includes calculating the color feature distance and calculating the deep learning high-dimensional feature distance. The likelihood functions of the passenger color feature distance and the deep learning high-dimensional feature distance are obtained by combining the color feature distance and the deep learning high-dimensional feature distance. Specifically, (1) Calculate the color feature distance. For a pair of boarding passenger i and disembarking passenger j, their color features are recorded as and The color feature distance d between the two c (i, j) can be expressed by Bhattacharyya distance: (2) Calculate the distance of deep learning high-dimensional features. The deep learning high-dimensional features of passenger i getting on the bus and passenger j getting off the bus are recorded as and The distance between the two is d F (i, j), calculated using Manhattan distance, is expressed as: (3) Fusion of passenger appearance feature distance likelihood function; Through (1) and (2), the likelihood function of passenger color feature distance and deep learning high-dimensional feature distance can be obtained. The likelihood function of passenger appearance feature distance is composed of color feature distance d c (i, j) and deep learning high-dimensional feature distance d F The likelihood function of the two (i, j) is fused, as shown in formulas (3) and (4), which represents the likelihood function of the color feature distance, P(d C |δ ij =1) represents the likelihood function of deep learning high-dimensional feature distance, θ C represents the weight of the color feature, θ F Represents the weight of high-dimensional features in deep learning; (4) Calculation of passenger characteristic distance distribution; The fused passenger appearance feature distance distribution likelihood function is obtained from (3) and (4), which are the passenger correct matching likelihood function P(d ij |δ ij =1) and the passenger mismatch likelihood function P(d ij |δ ij =0), the prior probability function P(δ ij =1) is the historical OD probability of public transportation; P(d ij )=P(d ij |d ij =1)P(δ ij =1)+P(d ij d ij =0)P(δ ij =0) (5); P(δ ij =0)=1-P(δ ij =1) (6); According to the Bayesian method, the posterior probability is shown in formula (7). Formula (8) calculates the matching probability P of the boarding passenger i and the getting-off passenger j. ij : P ij =P(δ ij =1|d ij ) (8); S4, matching passengers getting on and off the bus, and then performing bipartite graph matching; Passenger boarding and alighting matching is a two-step process. First, the matching range of passengers needs to be determined, and then bipartite graph matching is performed. The bipartite graph contains the set of boarding passengers U and the set of disembarking passengers D. k and v represent the passenger boarding and disembarking stations, respectively. If passenger i boards the bus at station k, then the matching disembarking passenger j can only disembark after station k. The set of passengers that can match passenger i is denoted as j∈S(i), and the matching range is obtained. After determining the matching range, the passenger boarding and alighting matching is constructed as a bipartite graph matching problem, and the optimal matching is performed with the goal of maximizing the total matching probability, as shown in formula (9); formula (10) converts the decision variable δ ij Specified as a binary integer, it is 1 if i and j match, otherwise it is 0; Formula (11) ensures that any boarding passenger i can only match at most one disembarking passenger j; Formula (12) ensures that any disembarking passenger j can only match at most one boarding passenger i; S5. Allocate OD amount of unsuccessfully matched passengers; The boarding and alighting pairs of passengers with a higher matching probability are selected as the credible partial matching results. The remaining unmatched passengers are subjected to OD inference through the allocation algorithm. Based on the OD of the successfully matched passengers, the boarding and alighting passenger flows at each station corresponding to the unmatched passengers are calculated respectively, and the boarding and alighting passenger flows are used as the input flow of the allocation algorithm. S6. Estimating bus OD demand; Comprehensive successful matching P ij >δ passenger OD and the unsuccessful matching passenger OD in the second part obtained by the allocation algorithm to obtain the final passenger OD demand result L ij : L ij =S ij +X ij (13); Among them, δ is the passenger boarding and disembarking matching probability threshold, S ij To match the OD needs of successful passengers, X ij It is the OD demand of unmatched passengers.

2. The method for matching passengers getting on and off the bus and estimating OD demand based on bus door video according to claim 1, characterized in that: In step S1, the bus video from which the passenger OD needs to be extracted is input, and Yolov5 and DeepSORT are used to identify and track passengers to obtain the boarding and alighting information of each passenger.

Images