A regular bus passenger flow OD calculation method and device based on vehicle-mounted video and a medium

Abstract

The present application relates to a kind of conventional bus passenger flow OD calculation method based on vehicle video, device and medium;It relates to the public transport passenger flow monitoring technical field, the method includes: in bus door, off bus door layout camera, synchronously collect multiple data and unify time stamp;Set up boarding and alighting event determination rule;Respectively detect boarding and alighting event and extract feature;With the passenger pool in the vehicle, the off bus target feature set is matched;According to the matching result, update passenger pool, statistics passenger flow, match failure then re-match, finally output each site boarding and alighting passenger number and real-time passenger number in the vehicle;The present application is calibrated by multiple source data, custom rule filters effective boarding and alighting event, trajectory feature fusion improves accuracy, constructs closed passenger pool and reduces operation load;Using joint matching algorithm improves accuracy, establishes correction mechanism to form closed loop, relies on modular architecture and outputs accurate data, provides real-time data source for bus passenger flow analysis, helps fine operation adjustment.

Description

Technical Field

[0001] This invention relates to the field of public transportation passenger flow monitoring technology, specifically to a conventional bus passenger flow OD calculation method, device, and medium based on onboard video. Background Technology

[0002] Urban public transportation is a core component of the urban integrated transportation system. As the primary mode of daily travel for urban residents, the operational efficiency and service quality of conventional buses directly impact the overall operational level of urban transportation. Passenger flow origin-destination (OD) data, reflecting the distribution of bus passengers' origins and destinations, is crucial for bus operators in route planning, capacity allocation, and station optimization. It also serves as key support for urban traffic management departments in public transportation network layout and travel demand analysis. With the continuous advancement of intelligent urban transportation construction, technologies such as onboard sensing equipment, computer vision, and target detection and tracking are rapidly developing, providing a technological foundation for the intelligent calculation of bus passenger flow OD. Passenger flow detection methods based on onboard video are gradually replacing traditional manual statistical methods, becoming the mainstream approach.

[0003] Current technologies for calculating the origin-destination (OD) of passenger flow in conventional public transportation based on in-vehicle video still have many problems that need to be solved: (1) Lack of unified benchmark for timestamps of multi-source data: The collection timestamps of various types of data such as video stream, door status data, GPS / BeiDou positioning data are generated independently by their respective software systems. Without a unified time benchmark, there is a problem of inconsistent timestamps when multi-source data is fused and processed, which leads to deviations in the determination of getting on and off the vehicle.

[0004] (2) Lack of precise constraints in determining boarding and alighting events: Existing solutions mostly use a single line crossing detection or frame difference analysis method to determine boarding and alighting events, which lacks joint constraints of spatial area and time window, and is prone to misjudging irrelevant human actions during non-station stopping periods as valid boarding and alighting behaviors.

[0005] (3) Insufficient representation of passenger features: Existing solutions mostly use single-frame features or equal-weighted multi-frame feature fusion, without considering the actual state differences of passenger targets within video frames. When some frames are incomplete due to occlusion or blurring, the features of low-quality frames will dilute the contribution of high-quality frames, causing the fused feature vector to deviate from the true appearance representation of passengers, affecting the subsequent matching effect.

[0006] (4) The global search matching computation load is large and the misjudgment rate is high: Some existing solutions adopt the global passenger database search method in the feature matching stage. The computation load increases linearly with the passenger scale and the real-time performance is difficult to guarantee. At the same time, most solutions rely on a single visual feature for matching, lacking dual verification at the time sequence and public transport business level, resulting in a high misjudgment rate.

[0007] (5) Lack of a sound correction and fallback mechanism for matching failure: When the alighting target cannot be successfully matched with the passengers in the vehicle, most existing solutions lack a systematic multi-dimensional correction process, simply discarding or using the default statistical method, which makes it difficult to guarantee the integrity of passenger flow OD statistics.

[0008] Existing representative technologies, such as the bus video passenger OD analysis method disclosed in CN113255552B, although they realize passenger OD analysis using onboard video, have significant shortcomings in multi-source data timestamp accurate synchronization calibration, full-frame quality difference perception and dynamic weighted feature fusion, closed passenger pool construction, time-series-business constraint three-dimensional joint matching, and multi-dimensional correction closed-loop mechanism, making it difficult to meet the actual needs of refined management of bus operations. Summary of the Invention

[0009] To address the shortcomings of existing technologies, this invention provides a method, device, and medium for calculating the origin-destination (OD) of passenger flow in conventional public transportation based on in-vehicle video. Using in-vehicle video as the core, it synchronously collects multi-source operational data and unifies timestamps. It accurately determines boarding and alighting events through preset rules, extracts passenger and alighting target features by combining trajectory-level multi-frame feature fusion, constructs a vehicle-level closed in-vehicle passenger pool, and uses a joint matching algorithm of visual features and temporal-business constraints to complete feature matching within the pool. It performs multi-dimensional corrections for failed matches and then completes passenger flow statistics and updates the passenger pool based on the matching results. This invention solves the problems of high misjudgment rate and low processing efficiency in traditional technologies, improving the accuracy and real-time performance of passenger flow OD calculation. The device adopts a modular architecture, is compatible with existing public transportation hardware, and can be directly deployed and applied, providing reliable data support for public transportation operation scheduling and urban traffic management.

[0010] To achieve the above objectives, the present invention provides the following technical solution: A conventional bus passenger flow OD calculation method based on in-vehicle video, characterized by comprising: S1, Multi-source data synchronous acquisition: Full-coverage cameras are deployed at the bus boarding and alighting doors to synchronously acquire door area video streams, door status data, and GPS / BeiDou positioning data. Hardware synchronization pulses are used to achieve frame-level timestamp unification. S2, Boarding and alighting event rule setting: Preset a 0.5-meter door area detection range for boarding and alighting doors, formulate rules for judging cross-line trajectories, and only judge valid boarding and alighting events within the door opening window at the station stop; S3, boarding detection and feature fusion: Human body detection and tracking are performed on the boarding door video. After determining the boarding event, the trajectory-level multi-frame feature fusion is performed by dynamically allocating weights based on frame clarity and occlusion degree to generate passenger objects and store them in the vehicle's exclusive closed passenger pool. S4, Disembarkation Detection and Feature Extraction: Using the same algorithm framework and feature model as the boarding algorithm, the feature set of disembarkation targets is extracted; S5, Closed Pool Joint Matching: The alighting feature is jointly matched with the passenger pool in three dimensions: visual features, travel duration time sequence, and route business. The search is only performed within the in-vehicle pool and no global search is performed. S6, Matching Correction and Passenger Flow Statistics: If the match is successful, the passenger pool and passenger flow data are updated; if the match fails, it is rematched after multi-dimensional correction. If it still fails, it is handled according to the business rules as a fallback, and finally the station passenger flow and real-time number of people in the vehicle are output.

[0011] Furthermore, in step S1, each car door is equipped with 1-2 high-definition cameras, covering the outer waiting area, the door opening and closing area, and the inner door area; the door status is distinguished into four states: door start / end, door close start / end; the timestamp adopts hardware pulse synchronization consistent with the video frame interval.

[0012] Furthermore, in step S2, the boarding trajectory is a continuous crossing of the line from the outer side to the inner side, and the alighting trajectory is a continuous crossing of the line from the inner side to the outer side; the valid events are limited to the time period when the train stops at the station and the doors are open.

[0013] Furthermore, the trajectory-level multi-frame feature fusion formula in step S3 is as follows:

[0014] In the formula, The global appearance feature vector after the fusion of passenger movement trajectories is a 256-dimensional or 512-dimensional numerical vector. The number of valid video frames within the time window in which the passenger trajectory meets the crossing rules; For the first The feature weights of the frames are dynamically assigned based on the clarity, integrity, and degree of occlusion of the passenger targets within the frame. Clear, unobstructed frames have higher weights than occluded or blurred frames. For the first The single-frame global appearance feature vector of the passenger target in the frame is obtained by encoding the entire human body image frame by the feature extraction network, which comprehensively represents the global features of clothing color and body shape outline.

[0015] Furthermore, the expression for the three-dimensional joint matching in step S5 is as follows:

[0016] In the formula, For the target feature set of getting off the bus and the first passenger in the pool of passengers inside the bus Overall matching degree of each passenger object; is a cosine similarity calculation function used to solve the global appearance feature similarity between the disembarking target and the passenger object, with a value range of [0, 1]. This is an indicator function; it outputs 1 if the condition inside the parentheses is true, and 0 if it is false. For the timestamp of the passenger's boarding event. The timestamp of the disembarkation event for the disembarking target. The reasonable time sequence constraint interval for bus travel duration; For passenger boarding stations, Get off at the target stop Due to route service constraints, the drop-off point must be after the boarding point on the vehicle's operating route.

[0017] Furthermore, in step S6, if the overall matching degree is greater than or equal to the preset threshold, it is successful and the maximum value is taken from multiple candidates; if it fails, the features are re-fused, the weights are optimized, and the constraints are corrected before retrying. If it still fails, it is manually reviewed or the default statistics are used.

[0018] This invention also protects a conventional bus passenger flow OD calculation device based on in-vehicle video, comprising: a data acquisition module, an alighting and boarding event rule setting module, an alighting detection and feature extraction module, a closed passenger pool management module, an alighting detection and feature extraction module, a joint matching and verification module, a passenger flow statistics module, and a main control module; each module is electrically connected to the main control module and works together to execute the above methods.

[0019] Furthermore, the device specifically includes: The data acquisition module includes an upper door camera, an lower door camera, a door status sensor, a GPS / BeiDou positioning module, and an on-board industrial control computer. It is used to collect video streams from the upper and lower door areas, door status data, and vehicle positioning data, and to complete the unified synchronization of timestamps for multi-source data. The boarding and alighting event rule setting module is used to preset physical detection areas for boarding and alighting doors, formulate rules for determining crossing the line, and set time window constraints for triggering boarding and alighting events. The boarding event detection and feature extraction module is used to perform human detection and multi-target tracking on the boarding door video stream, determine boarding events, and extract global appearance features of passengers through a trajectory-level multi-frame feature fusion algorithm to generate passenger objects. The closed in-vehicle passenger pool management module is used to construct a vehicle-level closed in-vehicle passenger pool and perform operations such as adding, deleting, feature retrieval, and real-time data updates of passenger objects. The vehicle exit event detection and feature extraction module is used to perform human detection and multi-target tracking on the exit door video stream, determine the vehicle exit event, and extract the global appearance features of the exit target through a trajectory-level multi-frame feature fusion algorithm to generate an exit target feature set. The joint matching and verification module is used to employ visual features and temporal sequence. The business constraint joint matching algorithm completes the matching of disembarkation targets and passenger objects in a closed in-vehicle passenger pool, performs multi-dimensional correction on targets that fail to match, and re-matches them. The passenger flow statistics module is used to count the number of passengers getting on and off at each station, the real-time number of passengers in the vehicle, and the net flow at each station based on the matching results, and output passenger flow statistics data. The main control module is used to coordinate the working sequence of each module, realize data transmission, storage and scheduling, and serves as the core control unit of the device.

[0020] Finally, this invention protects a computer-readable storage medium for storing program code that, when executed by a processor, implements the above-described method.

[0021] Compared with the prior art, the technical solution of this application has the following beneficial effects: (1) Hardware synchronization pulse triggering timestamp unified mechanism: The hardware synchronization pulse triggering method is used to mark the video stream, vehicle status data and positioning data with unified timestamps. The synchronization pulse interval is consistent with the video frame interval. The vehicle industrial control computer hardware clock is the only clock source, which eliminates the problem of inconsistent timestamps caused by the operating system scheduling delay of each data source in the software timestamp scheme, and provides a millisecond-level synchronized data foundation for the accurate determination of subsequent getting on and off the vehicle.

[0022] (2) Dual-constraint effective vehicle entry and exit event determination mechanism: Combining the spatial constraints of the preset physical detection area and the time constraints of the door opening time window, the mechanism filters irrelevant human actions in the door area from both spatial and temporal dimensions through logical operations, eliminating invalid data interference from the source and significantly improving the accuracy of vehicle entry and exit event determination.

[0023] (3) Trajectory-level multi-frame dynamic weighted feature fusion algorithm: In view of the actual state differences of passenger targets in video frames, feature weights are dynamically allocated according to the clarity, integrity and occlusion of the targets in the frame. The weight of clear and unoccluded frames is higher than that of frames with partial occlusion or blurriness, so that the fused feature vector is dominated by high-quality frames. This effectively solves the problem of low-quality frames diluting the features of high-quality frames in equal weight fusion and significantly improves the feature representation ability.

[0024] (4) Vehicle-level closed passenger pool reduces computational load: A vehicle-level dedicated data pool is constructed to store only passenger objects currently in the vehicle. The matching search range is closed within the vehicle-level space, eliminating the global crowd search step, effectively reducing computational load and eliminating cross-vehicle matching errors.

[0025] (5) Visual features and time-series-business constraints three-dimensional joint matching algorithm: integrates three dimensions: visual appearance cosine similarity, travel time time-series constraints, and station order business constraints. By using time-series constraints, matching pairs with abnormal travel time are excluded, and by using station order constraints, reverse matching pairs where the alighting station is before the actual boarding station are excluded. This effectively avoids the misjudgment problem of matching with a single visual feature and greatly improves the matching accuracy.

[0026] (6) Multi-dimensional correction and fallback closed-loop mechanism: A multi-dimensional correction mechanism is established for the target of matching failure. The system is systematically corrected from three dimensions: feature quality, weight allocation and constraint boundary before rematching, forming a complete "match-failure analysis-correction-rematch" closed loop; at the same time, a fallback scheme of manual review or default statistics is combined to ensure the integrity and reliability of passenger flow OD statistics. Attached Figure Description

[0027] Figure 1 The flowchart shows the conventional bus passenger flow OD calculation method based on in-vehicle video. Figure 2 This is a block diagram of the modular structure of a conventional bus passenger flow OD calculation device based on in-vehicle video. Figure 3 A schematic diagram of the hardware structure of an electronic device for calculating the origin-destination (OD) of bus passenger flow.

[0028] Wherein: 410, electronic equipment; 2001, first processor; 2002, memory; 2003, transceiver; 2004, second processor. Detailed Implementation

[0029] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0030] This invention presents a conventional bus passenger flow OD calculation method based on in-vehicle video. Relying on an integrated hardware architecture encompassing in-vehicle video acquisition, status perception, satellite positioning, and data processing, it integrates computer vision, multi-target tracking, trajectory-level feature fusion, and intelligent joint matching algorithms. This enables precise detection of boarding and alighting events at bus doors, effective extraction of individual passenger features, and refined statistics of passenger flow OD. It overcomes the limitations of traditional bus passenger flow statistics, which can only obtain the number of passengers boarding and alighting at stations and cannot match individual passenger trajectories. The corresponding in-vehicle video-based conventional bus passenger flow OD calculation device serves as the hardware and software carrier of this method. Through the coordinated operation of various functional modules, it completes the collection, processing, analysis, and statistics of multi-source operational data, ultimately outputting complete passenger flow OD-related data such as the number of passengers boarding and alighting at each station and the real-time number of passengers inside the vehicle. This provides accurate and real-time data support for bus network optimization, dynamic capacity allocation, and station resource planning. The method and device of this invention are adaptable to conventional buses of different models, are easy to deploy, and can achieve fully automated collection and statistics of passenger flow data during bus operation without manual intervention.

[0031] Specific implementation of conventional bus passenger flow OD calculation method based on in-vehicle video: The calculation method of this invention is implemented through six steps: synchronous acquisition of multi-source data, setting of rules for determining boarding and alighting events, detection and feature extraction of boarding events, detection and feature extraction of alighting events, joint matching within a closed passenger pool, and processing of matching results and passenger flow statistics. Figure 1 As shown, each step is interconnected, and the data and algorithms are linked layer by layer to ensure the accuracy and real-time performance of passenger flow OD statistics. The specific implementation methods for each step are as follows: S1, Multi-source data synchronous acquisition: The core implementation point of this step is to achieve full-area collection and timestamp unification of bus door area video streams, door status data, and vehicle positioning data, providing a synchronized and standardized data source for subsequent boarding and alighting event detection. Cameras are deployed at each boarding and alighting door of the bus using a full-area coverage approach, with 1-2 identical high-definition cameras configured for each door. The camera installation angles precisely cover three areas: the outer waiting area, the door opening / closing area, and the inner door area, ensuring no blind spots and fully capturing the entire trajectory of passengers boarding and alighting. Simultaneously, door status sensors are deployed at the doors, and a GPS / BeiDou dual-mode positioning module is installed on the vehicle's center console. Using an onboard industrial control computer as the hardware core, the system simultaneously collects full-area video streams from the boarding and alighting doors, door status data output from the door status sensors, and vehicle positioning data output from the GPS / BeiDou positioning module. Among them, the door status sensor can accurately identify and collect four types of door status data: door opening start, door opening position, door closing start, and door closing position, providing real-time feedback on the door's operating status; the GPS / BeiDou positioning module collects data at a frequency of 1Hz or higher, accurately obtaining the vehicle's real-time latitude and longitude, driving speed, and station location information, providing a basis for station matching for boarding and alighting events; the unified timestamp marking is based on the hardware clock of the on-board industrial control computer and is implemented using a hardware synchronization pulse triggering method. The interval of the synchronization pulse is consistent with the frame interval of the video stream, marking the collected video stream, door status data, and positioning data with a unique and unified timestamp, completely eliminating time deviations between multi-source data.

[0032] S2, Rules for determining boarding and alighting events: This step establishes standardized physical areas and behavioral rules for the accurate determination of boarding and alighting events, while also setting time constraints on event validity to prevent invalid human actions during non-stop phases from being mistakenly identified as boarding or alighting events. Dedicated physical detection areas are preset for the boarding and alighting doors. The preset physical detection area for the boarding door is a range from 0.5 meters outside to 0.5 meters inside the door, while the preset physical detection area for the alighting door is a range from 0.5 meters inside to 0.5 meters outside the door. These areas are set based on the width of a typical bus door and the typical movement trajectories of passengers boarding and alighting, and can be slightly adjusted to accommodate different bus models. Establish corresponding rules for determining crossing boundaries. The rule for determining crossing boundaries when boarding is that the passenger's movement trajectory crosses the boundary line from the outer area of ​​the boarding door into the inner area and the trajectory is continuous and uninterrupted. The rule for determining crossing boundaries when alighting is that the passenger's movement trajectory crosses the boundary line from the inner area of ​​the alighting door into the outer area and the trajectory is continuous and uninterrupted. The trigger time for boarding and alighting events is defined as the timestamp when the passenger's movement trajectory meets the above crossing rules. At the same time, event validity constraints are set. Only when the trigger time falls within the door opening time window of the bus stop is the corresponding human action determined as a valid boarding or alighting event. The door opening time window is defined by the door opening time and closing start time in the door status data.

[0033] S3, Boarding Event Detection and Feature Extraction: This step relies on computer vision algorithms to detect valid boarding events at the boarding door, and extracts the global appearance features of passengers through feature fusion algorithms to generate standardized passenger objects and incorporate them into a dedicated closed passenger pool within the vehicle. First, the video stream captured by the boarding door camera is processed in real time. A mature object detection algorithm is used to identify human targets in the video stream. Then, a multi-target tracking algorithm assigns a unique tracking ID to each detected human target and generates a complete motion trajectory based on the continuous frame position changes of the human target. The generated motion trajectory is compared with the boarding crossing judgment rules set in S2. When the trajectory meets the rules and the corresponding event is a valid boarding event, the human action corresponding to the trajectory is determined to be a boarding event. Trajectory-level multi-frame feature fusion is performed on the passenger motion trajectory determined to be a boarding event. The global appearance features of the passenger are extracted through the trajectory-level multi-frame feature fusion algorithm. The mathematical expression of this algorithm is:

[0034] In the formula, The global appearance feature vector after the fusion of passenger movement trajectory is a 256-dimensional or 512-dimensional numerical vector, and the dimension can be selected according to the actual detection accuracy requirements; n is the number of effective video frames within the time window in which the passenger trajectory meets the crossing rule. The number of effective video frames is the actual number of recognizable frames after removing blurry and severely occluded frames. The feature weight of the i-th frame is dynamically assigned based on the clarity, integrity, and occlusion of the passenger target within the frame. Frames with clear and unoccluded images are assigned higher weights, while frames with occlusion, blurriness, or incomplete human targets are assigned lower weights. The weight assignment range is 0-1. The fused global appearance feature vector for the passenger target in frame i is obtained by encoding the entire human body image bounding box using a pre-trained feature extraction network. It comprehensively represents the passenger's global appearance features such as clothing color, body shape, and height proportions. Based on the extracted fused global appearance feature vector, a standardized passenger object is generated. The passenger object includes a tracking ID and the fused global appearance feature vector. The system collects six categories of data: boarding station number, boarding event timestamp, boarding door area trajectory features, and effective frame count information from feature fusion. The boarding station number is determined by the real-time station location of the vehicle matched with GPS / BeiDou positioning data. The generated passenger objects are stored in real time in the closed passenger pool of the current vehicle. This closed passenger pool is a vehicle-level exclusive data pool that only stores passenger objects currently in the vehicle. It supports adding, deleting, and feature-based retrieval operations for passenger objects, and the data is updated in real time as the vehicle operates.

[0035] S4, Disembarkation Event Detection and Feature Extraction: The algorithm framework in this step is consistent with that for boarding events, realizing the detection of valid alighting events at the exit door and the extraction of global appearance features of the alighting target, generating an alighting target feature set to provide a foundation for subsequent passenger matching. It also ensures consistency in feature encoding between the boarding and alighting doors, avoiding matching errors caused by model differences. The video stream captured by the exit door camera undergoes the same processing procedure as the boarding door, using the same target detection algorithm to identify human targets in the video stream. The same multi-target tracking algorithm assigns temporary tracking IDs to detected alighting human targets and generates continuous motion trajectories based on the continuous frame position changes of the human targets. The generated motion trajectories are compared with the alighting crossing judgment rules set in S2. When the trajectory meets the rules and the corresponding event is a valid alighting event, the human action corresponding to that trajectory is determined to be an alighting event. The trajectory of the passenger determined to be an alighting event is processed using the same trajectory-level multi-frame feature fusion algorithm as the boarding event, extracting the global appearance features of the alighting target. The feature extraction network uses the same pre-trained model as the boarding door to ensure consistency in feature encoding rules between the boarding and alighting doors. The extracted fused global appearance feature vector is... The dimensions are consistent with the FT of the boarding passengers. A disembarking target feature set is generated based on the extracted fused global appearance feature vector. The disembarking target feature set includes the temporary tracking ID and the fused global appearance feature vector. The data consists of six categories: drop-off station number, drop-off event timestamp, drop-off door area trajectory features, and effective frame count information of feature fusion. The drop-off station number is determined by the real-time station location of the vehicle matched with GPS / BeiDou positioning data. The drop-off target features extracted in this step are only used for feature matching in the closed passenger pool of the current vehicle and do not perform feature retrieval of the global population, which effectively reduces the amount of computation and improves matching efficiency.

[0036] S5, joint matching within a closed passenger pool: This step employs a joint matching algorithm combining visual features and temporal-business constraints to accurately match the disembarking target feature set with passenger objects within a closed, vehicle-level passenger pool. Invalid matches are eliminated through multi-dimensional constraints, improving matching accuracy. The matching operation is performed only within the current vehicle's closed passenger pool, without cross-vehicle or cross-group global searches. A one-to-one feature matching and constraint verification is performed between the disembarking target feature set and each passenger object in the closed passenger pool. The combined matching degree is calculated using the joint matching algorithm of visual features and temporal-business constraints. The mathematical expression of this algorithm is:

[0037] In the formula, This represents the overall matching degree between the disembarking target feature set and the k-th passenger object in the passenger pool inside the vehicle. The larger the value, the higher the degree of matching between the two. This is a cosine similarity calculation function used to solve the global appearance feature similarity between the disembarking target and the passenger object. The value range is [0, 1], and the closer the value is to 1, the higher the visual feature matching degree. This is an indicator function that outputs 1 when the condition within the parentheses is true and 0 when the condition is false, used to implement the validity verification of timing and business constraints; For the timestamp of the passenger's boarding event. The timestamp of the disembarkation event for the target disembarkation target. The reasonable time sequence constraint interval for bus travel time is preset according to the characteristics of bus route operation, and can be adjusted in a personalized manner according to the station spacing and travel speed of different routes. For passengers to board the train, The stop to get off at. For route service constraints, the drop-off station must be after the boarding station on the vehicle's operating route to exclude reverse station matching errors caused by similar features.

[0038] S6, Matching Result Processing and Passenger Flow Statistics: This step performs standardized matching judgment, passenger pool update, and passenger flow statistics operations based on the overall matching results of S5. For disembarkation targets that failed to match, multi-dimensional correction and re-matching are performed to maximize the matching success rate, ultimately outputting complete passenger flow OD statistics. First, a preset overall matching threshold is established. This threshold can be adjusted according to the detection accuracy requirements of actual bus operation scenarios, based on the overall matching degree between the disembarking target and a certain passenger. If a match is found between the two passengers, it is determined that the two are successfully matched. If multiple passenger objects meet the matching criteria, the passenger object with the highest overall matching score is selected as the final matching result. After a successful match, the passenger object is removed from the closed passenger pool in real time, and passenger flow statistics are updated according to the matching result. These statistics include the number of passengers getting off at the alighting station, the net passenger flow at that station, and the real-time number of passengers inside the vehicle. The real-time number of passengers inside the vehicle is the number of remaining passenger objects in the passenger pool. The matching score between the alighting target and all passenger objects in the passenger pool is satisfied. If a match fails, the disembarking target is added to the pending confirmation queue. After performing multi-dimensional correction, a new match is performed in the passenger pool. The multi-dimensional correction includes re-fusion of multi-frame features, optimization of trajectory feature weight allocation, and verification of time-series business constraints. These methods optimize the feature set and constraints of the disembarking target, improving the success rate of re-matching. If the disembarking target still fails to match after multiple multi-dimensional corrections, it is processed according to the standard business rules of the public transportation industry. The matching result can be confirmed manually or by default, ensuring the integrity of passenger flow data. After all the above processing, the device outputs real-time complete passenger flow OD statistics, including the number of passengers boarding and alighting at each station and the real-time number of passengers inside the vehicle during its journey. The data supports local storage and remote transmission to the public transportation operation management platform.

[0039] Specific implementation of a conventional bus passenger flow OD calculation device based on in-vehicle video: The conventional bus passenger flow OD calculation device based on in-vehicle video of the present invention serves as the hardware and software implementation carrier for the aforementioned calculation method. This device includes a data acquisition module, a boarding / alighting event rule setting module, a boarding event detection and feature extraction module, a closed in-vehicle passenger pool management module, a disembarking event detection and feature extraction module, a joint matching and verification module, a passenger flow statistics module, and a main control module. Each module is electrically connected to the main control module, such as... Figure 2 As shown, the main control module coordinates the work sequence of each module through unified scheduling, completing the entire process of passenger flow OD calculation. The specific implementation methods and functions of each module are as follows: Data acquisition module The data acquisition module is the basic hardware module of the device, including the upper door camera, the lower door camera, the door status sensor, the GPS / BeiDou positioning module and the vehicle-mounted industrial control computer. Its core function is to complete the full-domain acquisition of video streams from the upper and lower door areas, door status data and vehicle positioning data, and to achieve unified synchronization of timestamps for multi-source data. The upper and lower door cameras are both high-definition industrial cameras, with 1-2 cameras per door. They are installed to cover the entire door area, eliminating blind spots. The door status sensors are contact or non-contact sensors, located on the door hinges or locks, accurately collecting four types of door status data: door opening start, door opening position, door closing start, and door closing position. The GPS / BeiDou positioning module is a dual-mode positioning module, located on the vehicle's center console, capable of acquiring real-time positioning data such as the vehicle's latitude, longitude, and station location. The onboard industrial control computer is the core of the module, with a built-in high-performance processor and storage unit. Using a hardware clock as a reference, it marks the output data of each acquisition device with a unified timestamp through hardware synchronization pulse triggering. The synchronization pulse interval is consistent with the video stream frame interval, completely eliminating time deviations from multi-source data.

[0040] Boarding and alighting event rule setting module The boarding and alighting event rule setting module is a software algorithm module deployed in the onboard industrial control computer's software system. Its core functions are to preset physical detection areas for boarding and alighting doors, formulate rules for determining crossing boundaries, and set time window constraints for triggering boarding and alighting events, providing standardized rules for accurate determination of subsequent boarding and alighting events. This module supports online adjustment and personalized configuration of rule parameters. It can adjust the range of the physical detection area according to the door width of different bus models, adjust the door opening time window according to the stop characteristics of different routes, and optimize the details of the crossing boundary determination rules according to the characteristics of passenger boarding and alighting actions. All set rules and parameters are stored in the module's local database and can be retrieved and executed in real time by the boarding event detection and feature extraction module and the alighting event detection and feature extraction module.

[0041] Boarding event detection and feature extraction module The boarding event detection and feature extraction module is the core algorithm module, deployed on the onboard industrial control computer. Its core functions are to perform human detection and multi-target tracking on the video stream from the boarding door, accurately determine valid boarding events, and extract the global appearance features of passengers through a trajectory-level multi-frame feature fusion algorithm to generate standardized passenger objects. This module integrates mature target detection algorithms, multi-target tracking algorithms, and trajectory-level multi-frame feature fusion algorithms. It can perform real-time frame processing on the video stream from the boarding door camera, assign a unique tracking ID to each human target and generate a continuous motion trajectory. After comparing the trajectory with preset rules, it determines a valid boarding event. For the passenger trajectory of a valid boarding event, the module will automatically extract multi-frame single-frame features and perform dynamic weight fusion to generate a 256-dimensional or 512-dimensional fused global appearance feature vector. Finally, it integrates all relevant data to generate standardized passenger objects and transmits the passenger objects to the closed in-vehicle passenger pool management module in real time.

[0042] Enclosed Passenger Pool Management Module The closed-loop passenger pool management module is a data storage and management module built on the local high-speed storage unit of the onboard industrial control computer. Its core function is to construct a vehicle-level closed-loop passenger pool and perform operations such as adding, deleting, feature retrieval, and real-time data updates for passenger objects. This module configures an independent vehicle-level dedicated data pool for each bus, storing only passenger objects currently inside the bus and not interacting with passenger pools of other vehicles. The module incorporates a highly efficient feature indexing algorithm, enabling rapid feature retrieval based on the global appearance features of passenger objects, providing high-speed data query support for subsequent joint matching. When it receives a passenger object from the boarding event detection and feature extraction module, the module automatically performs an addition operation. When it receives a successful matching signal from the joint matching and verification module, the module automatically performs a deletion operation for the corresponding passenger object, ensuring that the passenger pool data is consistent with the actual passenger situation inside the vehicle in real time.

[0043] Disembarkation event detection and feature extraction module The vehicle exit event detection and feature extraction module shares the same architecture and algorithm as the vehicle entry event detection and feature extraction module, deployed on the onboard industrial control computer. Its core functions include performing human detection and multi-target tracking on the exit door video stream, accurately determining valid exit events, and extracting the global appearance features of the exiting target through a trajectory-level multi-frame feature fusion algorithm to generate a standardized exit target feature set. This module employs the exact same target detection, multi-target tracking, and trajectory-level multi-frame feature fusion algorithms as the vehicle entry event detection and feature extraction module, and the feature extraction network also uses the same pre-trained model to ensure consistent feature encoding rules for the entry and exit doors. The module assigns a temporary tracking ID to the exiting human target and extracts the fused global appearance feature vector after determining a valid exit event. The system integrates all relevant data to generate a feature set of disembarking targets and transmits the feature set to the joint matching and verification module in real time. At the same time, the module only initiates matching requests to the closed passenger pool of the current vehicle and does not perform feature retrieval of the global population.

[0044] Joint matching and verification module: The joint matching and verification module is the algorithm execution module, deployed on the vehicle-mounted industrial control computer. Its core function is to use a joint matching algorithm based on visual features and temporal-business constraints to accurately match the feature set of disembarking targets with passenger objects within a closed passenger pool. For targets that fail to match, it performs multi-dimensional corrections and re-matches. This module receives the feature set of disembarking targets transmitted by the disembarking event detection and feature extraction module in real time, and simultaneously retrieves passenger object data from the closed passenger pool management module. It performs one-to-one joint matching within the passenger pool, calculating the comprehensive matching degree of each pair of matched objects. After comparing the comprehensive matching degree with a preset threshold, the module sends a matching success or failure signal to the closed passenger pool management module. For disembarking targets that fail to match, the module adds them to the confirmation queue and automatically performs multi-dimensional correction operations such as re-fusion of multi-frame features, optimization of trajectory feature weights, and verification of temporal-business constraint boundary values. After optimization, it re-initiates matching until a match is successful or the preset number of corrections is completed.

[0045] Passenger flow statistics module: The passenger flow statistics module is a data statistics and output module deployed on the onboard industrial control computer. Its core function is to calculate passenger flow data in real time, such as the number of passengers boarding and alighting at each station, the real-time number of passengers inside the vehicle, and the net passenger flow at each station, based on the matching results of the joint matching and verification module, and then output the statistical data in a standardized format. This module receives the matching results from the joint matching and verification module in real time, automatically accumulates the number of passengers boarding and alighting at each station, calculates the net passenger flow at each station, and updates the real-time number of passengers inside the vehicle based on real-time data from the closed passenger pool. The module encapsulates all statistical data in a standardized format, supporting local storage on the onboard industrial control computer's storage unit, and can also remotely transmit data to the public transport operation management platform via a wireless communication module, achieving cloud synchronization and real-time monitoring of passenger flow data. The output frequency of statistical data can be adjusted according to needs, supporting multi-dimensional output by station, by time, and other dimensions.

[0046] Main control module: The main control module is the central control unit of the device, built using a high-performance embedded processor and deployed on the vehicle-mounted industrial computer. Its core function is to coordinate the working sequence of each module, enabling efficient data transmission, storage, and scheduling between modules, and providing stable hardware resources and software operating environments for each module. The main control module establishes electrical connections with all other modules, allocates hardware computing and storage resources to each module, and sets the working sequence of each module according to the passenger flow OD calculation process to ensure that data transmission between modules is delayed and without loss. The module has built-in fault self-checking and status feedback functions, which can monitor the operating status of each module in real time. When a module malfunctions or operates abnormally, the main control module will promptly issue a fault warning signal, store and upload the fault information, and activate backup computing schemes to ensure the normal operation of the device's basic functions. The main control module supports remote parameter configuration, allowing bus operation managers to remotely adjust the module's working parameters through the backend platform to adapt to different operating scenarios.

[0047] Example of an electronic device for calculating the origin-destination (OD) of public transport passenger flow: Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention, such as... Figure 3 As shown. Optionally, the electronic device 410 may include a first processor 2001.

[0048] Optionally, the electronic device 410 may also include a memory 2002 and a transceiver 2003.

[0049] The first processor 2001, memory 2002, and transceiver 2003 can be connected via a communication bus.

[0050] The following is combined with Figure 3 A detailed description of each component of electronic device 410 is provided below: The first processor 2001 is the control center of the electronic device 410. It can be a single processor or a collective term for multiple processing elements. For example, the first processor 2001 can be one or more central processing units, a specific integrated circuit, or one or more integrated circuits configured to implement embodiments of the present invention, such as one or more microprocessors, or one or more field-programmable gate arrays.

[0051] Optionally, the first processor 2001 can perform various functions of the electronic device 410 by running or executing software programs stored in the memory 2002 and calling data stored in the memory 2002.

[0052] In a specific implementation, as one example, the first processor 2001 may include one or more CPUs, for example... Figure 3 CPU0 and CPU1 are shown in the diagram.

[0053] In a specific implementation, as one example, the electronic device 410 may also include multiple processors, for example... Figure 3 The first processor 2001 and the second processor 2004 are shown in the diagram. Each of these processors may be a single-core processor or a multi-core processor. Here, "processor" may refer to one or more devices, circuits, and / or processing cores used for processing data.

[0054] The memory 2002 is used to store the software program that executes the present invention, and is controlled by the first processor 2001 to execute it. The specific implementation method can be referred to the above method embodiment, and will not be repeated here.

[0055] Optionally, the memory 2002 may be a read-only memory or other type of static storage device capable of storing static information and instructions, a random access memory or other type of dynamic storage device capable of storing information and instructions, or an electrically erasable programmable read-only memory, a read-only optical disc or other optical disc storage, a CD storage, a magnetic disk storage medium or other magnetic storage device, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory 2002 may be integrated with the first processor 2001 or may exist independently and be connected via the interface circuit of the electronic device 410. Figure 3 (Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.

[0056] The transceiver 2003 is used to communicate with network devices or with terminal devices.

[0057] Alternatively, transceiver 2003 may include a receiver and a transmitter. Figure 3 (Not shown separately). The receiver is used to implement the receiving function, and the transmitter is used to implement the transmitting function.

[0058] Optionally, the transceiver 2003 can be integrated with the first processor 2001, or it can exist independently and be connected via the interface circuit of the electronic device 410. Figure 3 (Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.

[0059] It should be noted that, Figure 3 The structure of the electronic device 410 shown does not constitute a limitation on the router. Actual knowledge structure identification devices may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0060] Furthermore, the technical effect of the electronic device 410 can be referred to the technical effect of a conventional bus passenger flow OD calculation method based on vehicle video described in the above method embodiment, and will not be repeated here.

[0061] It should be understood that the first processor 2001 in the embodiments of the present invention can be a central processing unit, and the processor can also be other general-purpose processors, digital signal processors, application-specific integrated circuits, off-the-shelf programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0062] It should also be understood that the memory in the embodiments of the present invention may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory may be read-only memory, programmable read-only memory, erasable programmable read-only memory, electrically erasable programmable read-only memory, or flash memory. The volatile memory may be random access memory, which serves as an external cache. By way of example, but not limitation, many forms of random access memory are available, such as static random access memory, dynamic random access memory, synchronous dynamic random access memory, double data rate synchronous dynamic random access memory, enhanced synchronous dynamic random access memory, synchronously linked dynamic random access memory, and direct memory bus random access memory.

[0063] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0064] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0065] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0066] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0067] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A conventional bus passenger flow OD calculation method based on in-vehicle video, characterized in that, include: S1, Multi-source data synchronous acquisition: Full-coverage cameras are deployed at the bus boarding and alighting doors to synchronously acquire door area video streams, door status data, and GPS / BeiDou positioning data. Hardware synchronization pulses are used to achieve frame-level timestamp unification. S2, Boarding and alighting event rule setting: Preset a 0.5-meter door area detection range for boarding and alighting doors, formulate rules for judging cross-line trajectories, and only judge valid boarding and alighting events within the door opening window at the station stop; S3, boarding detection and feature fusion: Human body detection and tracking are performed on the boarding door video. After determining the boarding event, the trajectory-level multi-frame feature fusion is performed by dynamically allocating weights based on frame clarity and occlusion degree to generate passenger objects and store them in the vehicle's exclusive closed passenger pool. S4, Disembarkation Detection and Feature Extraction: Using the same algorithm framework and feature model as the boarding algorithm, the feature set of disembarkation targets is extracted; S5, Closed Pool Joint Matching: The alighting feature is jointly matched with the passenger pool in three dimensions: visual features, travel duration time sequence, and route business. The search is only performed within the in-vehicle pool and no global search is performed. S6, Matching Correction and Passenger Flow Statistics: If the match is successful, the passenger pool and passenger flow data are updated; if the match fails, it is rematched after multi-dimensional correction. If it still fails, it is handled according to the business rules as a fallback, and finally the station passenger flow and real-time number of people in the vehicle are output.

2. The method according to claim 1, characterized in that, In step S1, each car door is equipped with 1-2 high-definition cameras, covering the outer waiting area, the door opening and closing area, and the inner door area; the door status is distinguished into four states: door start / end, door close start / end; the timestamp uses hardware pulse synchronization consistent with the video frame interval.

3. The method according to claim 1, characterized in that, In step S2, the boarding trajectory is the outer side → inner side crossing the line continuously, and the alighting trajectory is the inner side → outer side crossing the line continuously; valid events are limited to the time period when the train stops at the station and the doors are open.

4. The method according to claim 1, characterized in that, The formula for trajectory-level multi-frame feature fusion in step S3 is: ； In the formula, The global appearance feature vector after the fusion of passenger movement trajectories is a 256-dimensional or 512-dimensional numerical vector. The number of valid video frames within the time window in which the passenger trajectory meets the crossing rules; For the first The feature weights of the frames are dynamically assigned based on the clarity, integrity, and degree of occlusion of the passenger targets within the frame. Clear, unobstructed frames have higher weights than occluded or blurred frames. For the first The single-frame global appearance feature vector of the passenger target in the frame is obtained by encoding the entire human body image frame by the feature extraction network, which comprehensively represents the global features of clothing color and body shape outline.

5. The method according to claim 1, characterized in that, The expression for the three-dimensional joint matching in step S5: ； In the formula, For the target feature set of getting off the bus and the first passenger in the pool of passengers inside the bus Overall matching degree of each passenger object; is a cosine similarity calculation function used to solve the global appearance feature similarity between the disembarking target and the passenger object, with a value range of [0, 1]. This is an indicator function; it outputs 1 if the condition inside the parentheses is true, and 0 if it is false. For the timestamp of the passenger's boarding event. The timestamp of the disembarkation event for the target disembarkation target. The reasonable time sequence constraint interval for bus travel duration; For passenger boarding stations, Get off at your desired stop Due to route service constraints, the drop-off point must be after the boarding point on the vehicle's operating route.

6. The method according to claim 5, characterized in that, If the overall matching degree is greater than or equal to the preset threshold in step S6, the result is successful and the maximum value is selected from multiple candidates; if it fails, the features are re-fused, the weights are optimized, and the constraints are corrected before retrying. If it still fails, the result is manually reviewed or the default statistics are used.

7. A conventional bus passenger flow OD calculation device based on in-vehicle video, characterized in that, include: The system includes a data acquisition module, a boarding / alighting event rule setting module, a boarding detection and feature extraction module, a closed passenger pool management module, an alighting detection and feature extraction module, a joint matching and verification module, a passenger flow statistics module, and a main control module. Each module is electrically connected to the main control module and works together to execute the method described in any one of claims 1-6.

8. The apparatus according to claim 7, characterized in that, The device specifically includes: The data acquisition module includes an upper door camera, an lower door camera, a door status sensor, a GPS / BeiDou positioning module, and an on-board industrial control computer. It is used to collect video streams from the upper and lower door areas, door status data, and vehicle positioning data, and to complete the unified synchronization of timestamps for multi-source data. The boarding and alighting event rule setting module is used to preset physical detection areas for boarding and alighting doors, formulate rules for determining crossing the line, and set time window constraints for triggering boarding and alighting events. The boarding event detection and feature extraction module is used to perform human detection and multi-target tracking on the boarding door video stream, determine boarding events, and extract global appearance features of passengers through a trajectory-level multi-frame feature fusion algorithm to generate passenger objects. The closed in-vehicle passenger pool management module is used to construct a vehicle-level closed in-vehicle passenger pool and perform operations such as adding, deleting, feature retrieval, and real-time data updates of passenger objects. The vehicle exit event detection and feature extraction module is used to perform human detection and multi-target tracking on the exit door video stream, determine the vehicle exit event, and extract the global appearance features of the exit target through a trajectory-level multi-frame feature fusion algorithm to generate an exit target feature set. The joint matching and verification module is used to employ visual features and temporal sequence. The business constraint joint matching algorithm completes the matching of disembarkation targets and passenger objects in a closed in-vehicle passenger pool, performs multi-dimensional correction on targets that fail to match, and re-matches them. The passenger flow statistics module is used to count the number of passengers getting on and off at each station, the real-time number of passengers in the vehicle, and the net flow at each station based on the matching results, and output passenger flow statistics data. The main control module is used to coordinate the working sequence of each module, realize data transmission, storage and scheduling, and serves as the core control unit of the device.

9. A computer-readable storage medium storing program code, characterized in that, When the program code is executed by the processor, it implements the method described in any one of claims 1-6.

Images