A passenger behavior analysis system based on image recognition and trajectory tracking

Abstract

This invention relates to the field of passenger behavior analysis technology and discloses a passenger behavior analysis system based on image recognition and trajectory tracking. The system includes: an image acquisition unit that acquires carriage image data; a trajectory evaluation unit that identifies passenger targets in the carriage image data, constructs the current state vector and error covariance matrix, and generates prior state estimates and prior error covariance estimates; a trajectory tracking unit that acquires the passenger position coordinates and passenger motion state at the next moment, generates a trajectory correlation matrix, selects target observations, determines gain coefficients, and generates posterior state estimates and posterior error covariance estimates; a sequence generation unit that updates the state vector and error covariance matrix at the next moment to form a continuous trajectory sequence; and a behavior analysis unit that identifies whether abnormal behavior exists. This application achieves continuous tracking of passenger trajectories and improves the accuracy of abnormal behavior identification.

Description

Technical Field

[0001] This invention relates to the field of passenger behavior analysis technology, and more specifically, to a passenger behavior analysis system based on image recognition and trajectory tracking. Background Technology

[0002] With the continuous improvement of urban rail transit networks, subways have become an important component of urban public transportation systems. Due to the relatively enclosed space, dense passenger flow, and high mobility of subway carriages, abnormal behaviors such as passengers falling, lying down, unusual lingering, and violent conflicts are prone to occur during peak hours or in emergencies. If these abnormal behaviors are not detected and addressed promptly, they may pose safety hazards, disrupt train operations, and negatively impact passenger experience. Therefore, how to achieve automated and intelligent monitoring and identification of passenger behavior within subway carriages has become a crucial research direction in the field of intelligent rail transit.

[0003] Among existing patents, CN119028026A discloses a method and device for analyzing passenger behavior in subway cars based on deep learning. This method analyzes surveillance video from the car, uses object detection models (such as YOLOv8) to identify human bodies, seats, and ground areas, extracts key point information of human bodies using pose estimation algorithms (such as OpenPose), constructs a skeleton sequence, and combines it with spatiotemporal graph convolutional networks (such as ST-GCN) to classify and identify passenger behavior, thereby enabling the judgment of abnormal behaviors such as lying down, falling, and disputes. However, this technology does not systematically model the positional changes of the same passenger in a continuous time series, making it difficult to maintain identity consistency when people are occluded, cross paths, or briefly leave the frame, easily leading to recognition interruptions or identity switching problems. In high-density car environments, passengers severely obstruct each other, and relying solely on key point or single-frame detection results can easily lead to misjudgments. Furthermore, it does not incorporate dynamic parameters such as continuous movement trajectories, speed changes, and direction changes for trend analysis, making it difficult to provide early warnings for behaviors such as violent movement, abnormal gathering, or prolonged lingering.

[0004] Therefore, it is necessary to design a passenger behavior analysis system based on image recognition and trajectory tracking to solve the problems existing in the current technology. Summary of the Invention

[0005] In view of this, the present invention proposes a passenger behavior analysis system based on image recognition and trajectory tracking, which aims to solve the problems of easy recognition interruption or identity switching, and difficulty in providing early warning of behaviors such as vigorous movement, abnormal gathering or long-term stay.

[0006] This invention proposes a passenger behavior analysis system based on image recognition and trajectory tracking, comprising: Image acquisition unit, used to acquire continuous time series of carriage image data; The trajectory evaluation unit is used to identify passenger targets in the carriage image data, determine the passenger position coordinates and passenger motion state at the current moment, construct the state vector and error covariance matrix at the current moment; perform linear transformation on the state vector at the current moment to generate prior state estimates, and iteratively update the error covariance matrix at the current moment to generate prior error covariance estimates. A trajectory tracking unit is used to acquire the passenger's position coordinates and motion state at the next moment as observations; determine the spatial distance between the prior state estimate and the observations, and construct a cost matrix; determine the optimal matching path in the cost matrix and generate a trajectory correlation matrix; select a target observation that matches the prior state estimate based on the trajectory correlation matrix; determine a gain coefficient based on the prior error covariance estimate; weight the difference between the target observation and the prior state estimate based on the gain coefficient, and add it to the prior state estimate to generate a posterior state estimate; update the prior error covariance estimate to generate a posterior error covariance estimate. The sequence generation unit is used to update the posterior state estimate to the state vector at the next time step, update the posterior error covariance estimate to the error covariance matrix at the next time step, and re-identify the passenger target to form a continuous trajectory sequence. The behavior analysis unit is used to identify whether there is abnormal behavior based on the changes in passenger position coordinates and motion state in the continuous trajectory sequence.

[0007] Furthermore, when constructing the current state vector and error covariance matrix, the trajectory evaluation unit includes: The trajectory evaluation unit preprocesses and extracts features from the carriage image data to locate the pixel region of the passenger target in the image; maps the center point of the pixel region to the passenger position coordinates in the carriage coordinate system, and determines the passenger motion state based on the coordinate displacement between consecutive frames; combines the passenger position coordinates and the passenger motion state to construct the state vector, constructs a square matrix with the same dimension as the state vector as the error covariance matrix, sets the diagonal elements as the initial variance of each component of the state vector, and sets the off-diagonal elements to zero.

[0008] Furthermore, when the trajectory evaluation unit generates the prior state estimate and the prior error covariance estimate, it includes: The trajectory evaluation unit projects the state vector at the current moment to the next moment based on the time step information to generate the prior state estimate; it also performs propagation deduction on the error covariance matrix at the current moment to generate the prior error covariance estimate.

[0009] Furthermore, when constructing the cost matrix, the trajectory tracking unit includes: The trajectory tracking unit evaluates the degree of deviation between the predicted position information and the measured position information in the prior state estimate and generates a distance cost value; the distance cost value of all the trajectories to be tracked is arranged and combined with all the observations to form the cost matrix.

[0010] Furthermore, when the trajectory tracking unit generates the trajectory correlation matrix, it includes: The trajectory tracking unit determines the matching relationship between the predicted trajectory and the observed value based on the positional differences in the cost matrix; and generates the trajectory association matrix based on the matching relationship.

[0011] Furthermore, when the trajectory tracking unit selects a target observation that matches the prior state estimate, it includes: The trajectory tracking unit traverses the trajectory association matrix, identifies the matrix elements marked as establishing association relationships and their corresponding row and column indices; and extracts the corresponding observation values ​​based on the row and column indices as the target observation values.

[0012] Furthermore, when determining the gain coefficient, the trajectory tracking unit includes: The trajectory tracking unit determines the gain coefficient based on the observed noise characteristics and the prior error covariance estimate.

[0013] Furthermore, when the trajectory tracking unit generates the posterior error covariance estimate, it includes: The trajectory tracking unit corrects the prior error covariance estimate based on the gain coefficient to generate the posterior error covariance estimate.

[0014] Furthermore, when the behavior analysis unit identifies whether abnormal behavior exists based on changes in passenger position coordinates and motion state in the continuous trajectory sequence, it includes: The behavior analysis unit derives the passenger's instantaneous motion trend parameters from the continuous trajectory sequence. The instantaneous motion trend parameters include instantaneous velocity magnitude, acceleration magnitude, and rate of change of motion direction. The instantaneous motion trend parameters are compared with a preset behavioral safety threshold range. If the instantaneous motion trend parameters exceed the behavioral safety threshold range, the current passenger is determined to be exhibiting abnormal violent movement behavior. The distribution density and dwell time of the passenger's position coordinates in the carriage space are analyzed. If the passenger's dwell time in the non-stop area exceeds a preset time limit or the position coordinates show a sudden change that matches the characteristics of a fall, the current passenger is determined to be exhibiting abnormal behavior of lingering or falling.

[0015] Furthermore, when the image acquisition unit acquires continuous time-series carriage image data, it includes: The image acquisition unit captures visual information inside the carriage; it adds a timestamp tag to each frame of visual information to establish a mapping relationship between the visual information and the continuous time series; it stores the visual information with the timestamp tag into a data cache queue and outputs the carriage image data of the continuous time series in chronological order.

[0016] Compared with existing technologies, the advantages of this invention are as follows: By constructing a state vector and an error covariance matrix, the position information and motion state of passengers are dynamically modeled, and a recursive mechanism combining prior prediction and observation update is adopted to achieve continuous tracking of passenger trajectories. Even in situations where there is obstruction by people, cross-walking, or short-term target loss within the carriage, trajectory continuity can still be maintained through state prediction, reducing trajectory drift and identity switching problems, thereby improving the stability of multi-target tracking. By constructing a spatial distance cost matrix between predicted states and observed values, and determining the optimal matching path to generate a trajectory association matrix, accurate data association is achieved in scenarios with multiple passengers, reducing the probability of false association in dense environments and improving tracking accuracy. The recursive correction of the state vector using a prediction-update mechanism, combined with gain calculation using the error covariance matrix while incorporating observed values, achieves suppression of measurement noise and environmental interference. Based on continuous time series trajectory data, the system comprehensively analyzes changes in passenger position and movement status. By continuously tracking movement trends, it can more accurately distinguish between normal rapid movement and abnormal violent movement, and identify behaviors such as falls, lingering or abnormal conflicts, thereby improving the accuracy and reliability of abnormal behavior identification. Attached Figure Description

[0017] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 This is a functional block diagram of a passenger behavior analysis system based on image recognition and trajectory tracking provided in an embodiment of the present invention. Detailed Implementation

[0018] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the disclosure to those skilled in the art. It should be noted that, unless otherwise specified, embodiments and features in the embodiments of the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0019] In some embodiments of this application, see Figure 1 As shown, a passenger behavior analysis system based on image recognition and trajectory tracking is proposed, including: Image acquisition unit, used to acquire continuous time series of carriage image data; The trajectory evaluation unit is used to identify passenger targets in the carriage image data, determine the passenger position coordinates and passenger motion state at the current moment, construct the state vector and error covariance matrix at the current moment; perform linear transformation on the state vector at the current moment to generate prior state estimates, and iteratively update the error covariance matrix at the current moment to generate prior error covariance estimates. The trajectory tracking unit is used to acquire the passenger's position coordinates and motion state at the next moment as observations; determine the spatial distance between the prior state estimate and the observations, and construct a cost matrix; determine the optimal matching path in the cost matrix and generate a trajectory correlation matrix; select a target observation that matches the prior state estimate based on the trajectory correlation matrix; determine the gain coefficient based on the prior error covariance estimate; weight the difference between the target observation and the prior state estimate based on the gain coefficient, and add it to the prior state estimate to generate a posterior state estimate; update the prior error covariance estimate and generate a posterior error covariance estimate. The sequence generation unit is used to update the posterior state estimate to the state vector at the next time step, update the posterior error covariance estimate to the error covariance matrix at the next time step, and re-identify the passenger target to form a continuous trajectory sequence. The behavior analysis unit is used to identify abnormal behavior based on changes in passenger position coordinates and motion state in a continuous trajectory sequence.

[0020] Specifically, the image acquisition unit is used to acquire continuous time series of carriage image data. It may include camera equipment installed on the top or side wall of the subway carriage to continuously capture images of the interior environment of the carriage at a preset frame rate, and add timestamp information to each frame of the image to ensure the order and continuity of the image data in the time dimension. The acquired image data can be output in chronological order through a buffer queue.

[0021] The trajectory evaluation unit is used to identify passenger targets in the carriage image data. Specifically, it can locate passengers in the image through target detection algorithms to obtain the image coordinates of each passenger at the current moment. It can also convert the pixel coordinates into position coordinates in the carriage coordinate system by combining the mapping relationship between pixel coordinates and carriage space. At the same time, it calculates the passenger's speed, direction and other motion state information by the position changes between consecutive frames. Further, it combines the passenger's position coordinates and motion state to form the current moment's state vector, and constructs an error covariance matrix that matches the dimension of the state vector to characterize the uncertainty of the state estimation. Subsequently, it performs a linear transformation on the current moment's state vector, that is, it projects the current state to the next moment through a preset state transition model to generate a priori state estimate. At the same time, it propagates and updates the error covariance matrix according to the state transition relationship to generate a priori error covariance estimate, thereby completing the prediction process.

[0022] The trajectory tracking unit acquires the passenger's position coordinates and motion state at the next moment as observations. Specifically, it obtains actual observation information by performing target recognition again on the next frame image. Then, it calculates the spatial distance between the prior state estimate and the observed values, and constructs a cost matrix based on the distance relationship between each predicted trajectory and the observed target. The optimal matching path is determined by globally optimizing the cost matrix, generating a trajectory correlation matrix to represent the matching relationship between the predicted trajectory and the observed target. Based on this trajectory correlation matrix, a target observation value matching the prior state estimate is selected. Furthermore, a gain coefficient is determined based on the prior error covariance estimate and observation noise parameters. This gain coefficient is used to weight and correct the difference between the target observation value and the prior state estimate, and the correction is added to the prior state estimate to generate a posterior state estimate. Simultaneously, the prior error covariance estimate is corrected and updated to generate a posterior error covariance estimate, thus completing one state update process.

[0023] The sequence generation unit is used to take the posterior state estimate as the state vector input for the next time step and the posterior error covariance estimate as the error covariance matrix input for the next time step, and continue to participate in the next round of prediction and update cycle. It also combines the newly acquired image data to re-identify passenger targets, thereby forming a continuous and stable passenger trajectory sequence in the time dimension.

[0024] The behavior analysis unit is used to comprehensively analyze the changes in passenger position coordinates and motion state in a continuous trajectory sequence. By extracting information such as trajectory change trends, speed change characteristics, abnormal changes in direction, or spatial dwelling characteristics, it determines whether there are abnormal behaviors, such as falls, lingering, or violent conflicts, thereby achieving intelligent identification and early warning of abnormal behaviors in the carriage environment.

[0025] Understandably, by combining image recognition with trajectory prediction and update mechanisms, a passenger state estimation model under continuous time series is constructed, enabling stable tracking of multiple passenger targets within the carriage and reducing trajectory drift and mismatch problems in complex scenarios such as personnel occlusion and cross-walking. The recursive approach of prior prediction and observation update improves the system's robustness to noise interference and environmental changes. Behavioral analysis based on continuous trajectory sequences can more accurately identify abnormal behaviors such as falls, lingering, or violent movements, improving the accuracy and real-time performance of anomaly detection.

[0026] In some embodiments of this application, when the trajectory evaluation unit constructs the current state vector and error covariance matrix, it includes: The trajectory evaluation unit preprocesses and extracts features from the carriage image data to locate the pixel region of the passenger target in the image; it maps the center point of the pixel region to the passenger position coordinates in the carriage coordinate system, and determines the passenger motion state based on the coordinate displacement between consecutive frames; it combines the passenger position coordinates and the passenger motion state to construct a state vector, constructs a square matrix with the same dimension as the state vector as the error covariance matrix, sets the diagonal elements as the initial variance of each component of the state vector, and sets the off-diagonal elements to zero.

[0027] Specifically, the image data of the carriage output by the image acquisition unit undergoes preprocessing, including image denoising, brightness equalization, scale normalization, and perspective distortion correction, to eliminate the impact of lighting variations and camera angle differences on recognition accuracy. Subsequently, feature extraction and target detection are performed on the preprocessed image. The target detection algorithm locates the pixel region of the passenger target in the image; the pixel region can be a detection box or a segmented contour region, used to characterize the spatial range of the passenger in the image plane. After obtaining the pixel region, the coordinates of the center point of the pixel region are extracted as the target position in the image coordinate system. Based on pre-calibrated camera intrinsic and extrinsic parameters, the image coordinates are mapped to the actual spatial coordinate system of the carriage through coordinate transformation, obtaining the physical position coordinates of the passenger in the carriage plane. Further, by comparing continuous... The passenger's position coordinate difference between consecutive time frames is used to calculate the displacement change per unit time, thereby determining the passenger's motion state information. The motion state can include velocity components, motion direction, or acceleration components. Subsequently, the passenger's position coordinates and motion state information are combined in a preset order to form the current state vector. For example, a column vector structure containing position and velocity components can be constructed. While constructing the state vector, a square matrix with the same dimension as the state vector is established as the error covariance matrix to characterize the uncertainty of the estimated values ​​of each state component. The diagonal elements of the error covariance matrix are set as the initial variance values ​​of each component of the state vector to represent the estimation error range of each state component, while the off-diagonal elements are initialized to zero, indicating that the state components are assumed to be independent and uncorrelated in the initial stage.

[0028] It is understandable that by constructing state vectors and error covariance matrices, unified modeling of passenger position and motion information is achieved; by coordinate mapping and continuous frame displacement calculation, spatial positioning accuracy and motion state estimation accuracy are improved; and the use of a diagonally initialized error covariance matrix structure helps to simplify the calculation process and enhance the initial stability of the system.

[0029] In some embodiments of this application, when the trajectory evaluation unit generates prior state estimates and prior error covariance estimates, it includes: The trajectory evaluation unit projects the state vector of the current moment to the next moment based on the time step information to generate a priori state estimate; it also propagates and extrapolates the error covariance matrix of the current moment to generate a priori error covariance estimate.

[0030] Specifically, the time step information between the current moment and the next moment is first obtained. The time step can be determined based on the image acquisition frame rate or timestamp data, and is used to characterize the actual time interval between two state updates. After obtaining the time step information, the trajectory evaluation unit performs a time projection operation on the current moment's state vector based on a preset state transition model. The state transition model is used to characterize the passenger's motion pattern within a short time interval. The projection process includes linearly extrapolating the position component in the current moment's state vector according to the product of the velocity component and the time step, while keeping the velocity component unchanged or updating it in combination with preset acceleration parameters, thereby forming a prediction of the state at the next moment. The system measures the state vector and uses the predicted result as a priori state estimate. After completing the state vector prediction, the trajectory evaluation unit performs propagation and deduction processing on the error covariance matrix corresponding to the current moment. Specifically, the error covariance matrix is ​​transformed according to the state transition model, so that the uncertainty of each state component is transmitted over time. At the same time, a process noise term is superimposed to compensate for the increase in uncertainty caused by system model error and environmental disturbance. Finally, the uncertainty matrix corresponding to the prediction stage is obtained and determined as the priori error covariance estimate. Through the above synchronous prediction processing of state vector and error covariance matrix, the continuous recursion of the system from the current moment to the next moment is realized.

[0031] Understandably, by introducing a state projection mechanism based on time steps, continuous prediction of passenger movement trends can be made between adjacent frames, enhancing the smoothness and coherence of the trajectory; by synchronously propagating and deriving the error covariance matrix, a quantitative description of prediction uncertainty is achieved.

[0032] In some embodiments of this application, the trajectory tracking unit constructs the cost matrix, including: The trajectory tracking unit assesses the degree of deviation between the predicted and measured position information based on the predicted position information in the prior state estimate and the measured position information in the observation, and generates a distance cost value; the distance cost values ​​of all tracks to be tracked are combined with all observation values ​​to form a cost matrix.

[0033] Specifically, firstly, the trajectory tracking unit reads the prior state estimate obtained in the previous prediction stage and extracts the predicted position information corresponding to each tracked trajectory from the prior state estimate. The predicted position information is the spatial coordinates of the next time step derived based on the state transition model. Simultaneously, it acquires the observation values ​​obtained through target detection at the current time and extracts the measured position information of each passenger target from the observation values. The measured position information is the actual position coordinates in the carriage coordinate system obtained by spatial mapping of image coordinates. Subsequently, it performs pairwise matching calculations on the predicted position information of each tracked trajectory and the measured position information of each observation value to evaluate the deviation between the predicted position information and the measured position information. The degree of deviation can be obtained by calculating Euclidean distance, Mahalanobis distance, or other spatial metrics. It is used to quantify the closeness between the predicted location and the actual observed location, and the calculation result is defined as the distance cost value. The smaller the distance cost value, the higher the matching probability. After obtaining the distance cost values ​​between all predicted trajectories and all observed targets, the distance cost values ​​corresponding to each tracked trajectory are arranged according to the observation target number. The distance cost values ​​of all tracked trajectories and all observation values ​​are arranged and combined to construct a two-dimensional matrix structure. The rows of the two-dimensional matrix represent each tracked trajectory, the columns represent each observation target, and the matrix elements are the corresponding distance cost values, thus obtaining the cost matrix.

[0034] Understandably, by quantifying the deviation between predicted and measured location information and constructing a unified cost matrix structure, a systematic matching expression between multi-target trajectories and multi-observation targets is achieved, reducing the error caused by manually setting matching rules and improving the accuracy and stability of multi-target association.

[0035] In some embodiments of this application, when the trajectory tracking unit generates the trajectory correlation matrix, it includes: The trajectory tracking unit determines the matching relationship between the predicted trajectory and the observed value based on the positional differences in the cost matrix; and generates a trajectory correlation matrix based on the matching relationship.

[0036] Specifically, based on the aforementioned constructed cost matrix, a comprehensive analysis is performed on the positional differences between the predicted trajectories and observed values ​​represented by each element in the matrix. The positional difference represents the distance cost between the predicted and measured positional information, reflecting the matching probability between the two. Subsequently, the trajectory tracking unit optimizes the cost matrix according to preset matching criteria. These criteria may include the minimum total cost principle, threshold filtering principle, or one-to-one correspondence constraint principle, ensuring that each predicted trajectory matches at most one observed value at any given time, and that each observed value is associated with only one predicted trajectory. In the specific implementation process... The cost matrix can be solved globally for the optimal path by traversal search or optimization allocation algorithm, thereby selecting the matching scheme with the minimum total cost among all possible matching combinations to determine the optimal matching relationship between the predicted trajectory and the observed value. After determining the matching relationship, the matching results are encoded in matrix form to construct a trajectory association matrix, where the rows of the trajectory association matrix correspond to the predicted trajectory number and the columns correspond to the observed target number. When a predicted trajectory matches an observed target, an association identifier value is assigned to the corresponding matrix position; otherwise, a value of zero or null is assigned. This forms a structured matrix data that can intuitively represent the correspondence between the trajectory and the observation.

[0037] Understandably, by performing global optimization matching on the cost matrix and constructing a trajectory correlation matrix, a standardized, one-to-one correspondence between multiple predicted trajectories and multiple observed targets is achieved, reducing the occurrence of mismatches and missed matches.

[0038] In some embodiments of this application, when the trajectory tracking unit selects a target observation that matches the prior state estimate, it includes: The trajectory tracking unit traverses the trajectory correlation matrix, identifies the matrix elements marked as establishing correlation relationships and their corresponding row and column indices; based on the row and column indices, it extracts the corresponding observation values ​​as target observation values.

[0039] Specifically, after constructing the trajectory association matrix, the trajectory tracking unit performs a traversal scan of the matrix. This scan can sequentially read each element in the matrix in a row-first or column-first manner to find elements marked as having established associations. The marked matrix element indicates a matching relationship between the predicted trajectory represented by the corresponding row and the observation target represented by the corresponding column. After identifying the marked element, the row and column indices of that element are further obtained. The row index indicates the corresponding prior state estimate number, and the column index indicates the corresponding observation value number. Subsequently, based on the column index, the unit performs index positioning within the current observation set and extracts the observation data corresponding to that column index. This observation data includes the passenger's measured position coordinates and motion state information. The extracted observation data is then identified as the target observation value matching the corresponding prior state estimate, thus achieving a correspondence between the predicted trajectory and the actual observation. Through the above traversal identification and index extraction process, the mapping transformation from the trajectory association matrix to specific target observation values ​​is completed.

[0040] Understandably, by traversing and identifying the trajectory correlation matrix and accurately extracting the target observation value based on the row and column index, a structured correspondence between the predicted trajectory and the actual observation data is achieved, avoiding errors caused by manual intervention or repeated matching.

[0041] In some embodiments of this application, the trajectory tracking unit determines the gain coefficient by including: The trajectory tracking unit determines the gain coefficient based on the observed noise characteristics and the prior error covariance estimate.

[0042] Specifically, after obtaining the target observation value that matches the prior state estimate, the trajectory tracking unit reads the corresponding prior error covariance estimate, which characterizes the uncertainty of each state component in the prediction stage. Simultaneously, it acquires the observation noise characteristic parameters, which describe the statistical properties of the sensor measurement error, including the variance and covariance structure of the observation error. The observation noise characteristics can be obtained through system calibration or historical data statistics and are represented as an observation noise covariance matrix. Subsequently, the trajectory tracking unit bases its analysis on the relationship between the prior error covariance estimate and the observation noise covariance matrix. The relative magnitude relationship is comprehensively processed to measure the credibility weighting between the prediction result and the observation data. When the prediction uncertainty is large and the observation noise is small, the system increases the weight of the observation value in the state update, and vice versa. Through the joint analysis of prediction uncertainty and observation uncertainty, the gain coefficient used for state update is calculated. The gain coefficient is used to adjust the weighted difference between the target observation value and the prior state estimate, thereby achieving optimal fusion of prediction information and observation information during the update process. In this way, the gain coefficient can dynamically reflect the current uncertainty distribution of the system.

[0043] Understandably, by combining the prior error covariance estimate with the observation noise characteristics to determine the gain coefficient, an adaptive weight allocation between the predicted and observed information is achieved, thereby improving the accuracy and stability of the state update process.

[0044] In some embodiments of this application, when the trajectory tracking unit generates the posterior error covariance estimate, it includes: The trajectory tracking unit corrects the prior error covariance estimate based on the gain coefficient and generates the posterior error covariance estimate.

[0045] Specifically, after calculating the gain coefficient, the trajectory tracking unit reads the prior error covariance estimate corresponding to the current moment. The prior error covariance estimate is used to describe the uncertainty distribution of each state component in the prediction stage. Subsequently, the prior error covariance estimate is corrected based on the gain coefficient. The correction process involves redistributing and compressing the uncertainty contained in the prediction stage after the observation update is completed. This is achieved by introducing the gain coefficient to perform matrix operations on the prediction error covariance matrix, thereby reducing the error components related to the observation information while retaining the uncertainty parts in the system model that have not yet been fully corrected by the observation. Specifically, the trajectory tracking unit introduces the gain coefficient into the error covariance update formula and performs matrix concatenation transformation on the prior error covariance estimate to form a new error covariance matrix structure, reflecting the state estimation error level after fusing the observation information. The error covariance matrix obtained after the above correction process is the posterior error covariance estimate, used to characterize the degree of uncertainty after the state update at the current moment and to provide an initial error basis for the state prediction at the next moment. In this way, a closed-loop recursion of the error covariance between the prediction and update stages is realized.

[0046] Understandably, by using the gain coefficient to correct the prior error covariance estimate, the uncertainty of the state estimate is effectively converged after fusing observation information, thus improving the accuracy of the error description.

[0047] In some embodiments of this application, when the behavior analysis unit identifies whether abnormal behavior exists based on changes in passenger position coordinates and motion state in a continuous trajectory sequence, it includes: The behavior analysis unit derives the instantaneous motion trend parameters of passengers from the continuous trajectory sequence. The instantaneous motion trend parameters include the instantaneous velocity magnitude, acceleration magnitude, and rate of change of motion direction. The instantaneous motion trend parameters are compared with the preset behavioral safety threshold range. If the instantaneous motion trend parameters exceed the behavioral safety threshold range, the current passenger is determined to be exhibiting abnormal violent movement behavior. The distribution density and dwell time of the passenger's position coordinates in the carriage space are analyzed. If the passenger's dwell time in the non-stop area exceeds the preset time limit or the position coordinates show a sudden change that matches the characteristics of a fall, the current passenger is determined to be exhibiting abnormal behavior of lingering or falling.

[0048] Specifically, the behavior analysis unit performs time series analysis on the continuous trajectory sequence output by the sequence generation unit, extracting the state changes between adjacent time nodes from each passenger trajectory. Based on this, instantaneous motion trend parameters are derived to characterize the passenger's current motion features. These instantaneous motion trend parameters include instantaneous velocity magnitude, acceleration magnitude, and rate of change of motion direction. Instantaneous velocity magnitude is calculated from the position displacement per unit time, acceleration magnitude is calculated from the velocity difference between adjacent time points and the time interval, and the rate of change of motion direction is quantified by the change in the angle between adjacent trajectory vectors, thus comprehensively reflecting the passenger's dynamic behavioral characteristics within a short time scale. After obtaining the instantaneous motion trend parameters, they are compared with a preset behavioral safety threshold range. This behavioral safety threshold range is pre-set based on the carriage operating environment, safety regulations, and historical statistical data, used to define the numerical range between normal and abnormal behavior. When instantaneous velocity is detected... If any parameter among magnitude, acceleration magnitude, or rate of change of motion direction exceeds the corresponding safety threshold range, the current passenger is determined to have exhibited abnormal violent movement behavior. Furthermore, the behavior analysis unit comprehensively analyzes the distribution density and dwell time of passenger position coordinates within the carriage space. Specifically, it determines whether abnormal lingering occurs by statistically analyzing the continuous dwell time of passengers in specific spatial areas, and identifies non-stopping areas based on carriage functional area division information. When a passenger's dwell time in a non-stopping area exceeds a preset time limit, it is determined to be abnormal lingering behavior. Simultaneously, the behavior analysis unit performs abrupt change detection on the trend of position coordinate changes in a continuous trajectory. When it detects a sudden drop in height, an abnormal decrease in displacement amplitude, and accompanying posture changes within a short period, it determines that the trajectory characteristics conform to a fall behavior pattern, thus classifying it as abnormal fall behavior. Through the above multi-parameter joint analysis mechanism, automatic identification of multiple types of abnormal behaviors, such as violent movement, abnormal lingering, and falls, is achieved.

[0049] Understandably, by extracting instantaneous motion trend parameters from continuous trajectory sequences and combining them with spatial dwell characteristics for comprehensive judgment, the accuracy and real-time performance of abnormal behavior identification are improved. By adopting a combination of multi-dimensional threshold comparison and mutation detection, the false alarm rate is reduced, and the ability to identify abnormal behavior and provide safety warnings in complex carriage environments is enhanced.

[0050] In some embodiments of this application, when the image acquisition unit acquires continuous time-series carriage image data, it includes: The image acquisition unit captures visual information inside the carriage; it adds a timestamp label to each frame of visual information to establish a mapping relationship between visual information and continuous time series; it stores the visual information with timestamp labels into a data buffer queue and outputs the continuous time series of carriage image data in chronological order.

[0051] Specifically, the image acquisition unit captures visual information within the carriage in real time using cameras installed inside the carriage. This visual information includes passenger activity, spatial structure, background, and changes in ambient lighting. The cameras can be fixedly installed on the ceiling or side walls of the carriage to ensure coverage of key activity areas and minimize blind spots. After acquiring a single frame, the image acquisition unit adds a timestamp to each frame, identifying the acquisition time. The timestamp can be generated based on the system's internal clock or an external synchronous clock to ensure the continuity and accuracy of time recording, thus establishing a mapping between each frame of visual information and a continuous time series, making the image data traceable in the time dimension. Subsequently, the timestamped visual information is written into a data buffer queue in the acquisition order. The data buffer queue temporarily stores the image data to be processed and maintains the temporal order of the data through a first-in, first-out (FIFO) queue mechanism. Finally, the image acquisition unit outputs the visual information data sequentially according to the timestamp order, forming a carriage image data stream with a strict time series structure.

[0052] Understandably, by attaching a timestamp to each frame of the image and establishing a cache queue management mechanism, the orderly organization of the carriage image data in the time dimension is achieved, which improves the continuity and accuracy of data processing and avoids the problem of timing disorder caused by data transmission delay or frame rate fluctuation.

[0053] In summary, by constructing state vectors and error covariance matrices, dynamic modeling of passenger position information and motion states is achieved. A recursive mechanism combining prior prediction and observation updates enables continuous tracking of passenger trajectories. Even in situations where there is obstruction by people, cross-walking, or short-term target loss within the carriage, trajectory continuity can be maintained through state prediction, reducing trajectory drift and identity switching issues, thereby improving the stability of multi-target tracking. By constructing a spatial distance cost matrix between predicted states and observed values, and determining the optimal matching path to generate a trajectory association matrix, accurate data association is achieved in scenarios with multiple passengers, reducing the probability of false associations in dense environments and improving tracking accuracy. A prediction-update mechanism is used to recursively correct the state vector, incorporating observations while performing gain calculations using the error covariance matrix, thus suppressing measurement noise and environmental interference. Based on continuous time-series trajectory data, comprehensive analysis of passenger position and motion state changes is performed. Through continuous tracking of motion trends, normal rapid movement and abnormal violent movement are more accurately distinguished, identifying behaviors such as falls, stagnation, or abnormal conflicts, improving the accuracy and reliability of abnormal behavior identification.

[0054] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A passenger behavior analysis system based on image recognition and trajectory tracking, characterized in that, include: Image acquisition unit, used to acquire continuous time series of carriage image data; The trajectory evaluation unit is used to identify passenger targets in the carriage image data, determine the current passenger position coordinates and passenger motion state, and construct the current state vector and error covariance matrix. A linear transformation is performed on the state vector at the current moment to generate a priori state estimate, and the error covariance matrix at the current moment is iteratively updated to generate a priori error covariance estimate. The trajectory tracking unit is used to acquire the passenger's position coordinates and motion state at the next moment as observation values; Determine the spatial distance between the prior state estimate and the observed value, and construct the cost matrix; The optimal matching path in the cost matrix is ​​determined, and a trajectory correlation matrix is ​​generated. Based on the trajectory correlation matrix, a target observation value that matches the prior state estimate is selected. A gain coefficient is determined based on the prior error covariance estimate. The difference between the target observation value and the prior state estimate is weighted according to the gain coefficient and superimposed on the prior state estimate to generate a posterior state estimate. The prior error covariance estimate is updated to generate a posterior error covariance estimate. The sequence generation unit is used to update the posterior state estimate to the state vector at the next time step, update the posterior error covariance estimate to the error covariance matrix at the next time step, and re-identify the passenger target to form a continuous trajectory sequence. The behavior analysis unit is used to identify whether there is abnormal behavior based on the changes in passenger position coordinates and motion state in the continuous trajectory sequence.

2. The passenger behavior analysis system based on image recognition and trajectory tracking according to claim 1, characterized in that, When constructing the current state vector and error covariance matrix, the trajectory evaluation unit includes: The trajectory evaluation unit preprocesses and extracts features from the carriage image data to locate the pixel region of the passenger target in the image; maps the center point of the pixel region to the passenger position coordinates in the carriage coordinate system, and determines the passenger motion state based on the coordinate displacement between consecutive frames; combines the passenger position coordinates and the passenger motion state to construct the state vector, constructs a square matrix with the same dimension as the state vector as the error covariance matrix, sets the diagonal elements as the initial variance of each component of the state vector, and sets the off-diagonal elements to zero.

3. The passenger behavior analysis system based on image recognition and trajectory tracking according to claim 2, characterized in that, When the trajectory evaluation unit generates prior state estimates and prior error covariance estimates, it includes: The trajectory evaluation unit projects the state vector at the current moment to the next moment based on the time step information to generate the prior state estimate; it also performs propagation deduction on the error covariance matrix at the current moment to generate the prior error covariance estimate.

4. The passenger behavior analysis system based on image recognition and trajectory tracking according to claim 1, characterized in that, When constructing the cost matrix, the trajectory tracking unit includes: The trajectory tracking unit evaluates the degree of deviation between the predicted position information and the measured position information in the prior state estimate and generates a distance cost value; the distance cost value of all the trajectories to be tracked is arranged and combined with all the observations to form the cost matrix.

5. The passenger behavior analysis system based on image recognition and trajectory tracking according to claim 4, characterized in that, When the trajectory tracking unit generates the trajectory correlation matrix, it includes: The trajectory tracking unit determines the matching relationship between the predicted trajectory and the observed value based on the positional differences in the cost matrix; and generates the trajectory association matrix based on the matching relationship.

6. The passenger behavior analysis system based on image recognition and trajectory tracking according to claim 5, characterized in that, When the trajectory tracking unit selects a target observation that matches the prior state estimate, it includes: The trajectory tracking unit traverses the trajectory association matrix, identifies the matrix elements marked as establishing association relationships and their corresponding row and column indices; and extracts the corresponding observation values ​​based on the row and column indices as the target observation values.

7. The passenger behavior analysis system based on image recognition and trajectory tracking according to claim 6, characterized in that, When the trajectory tracking unit determines the gain coefficient, it includes: The trajectory tracking unit determines the gain coefficient based on the observed noise characteristics and the prior error covariance estimate.

8. The passenger behavior analysis system based on image recognition and trajectory tracking according to claim 7, characterized in that, When the trajectory tracking unit generates the posterior error covariance estimate, it includes: The trajectory tracking unit corrects the prior error covariance estimate based on the gain coefficient to generate the posterior error covariance estimate.

9. The passenger behavior analysis system based on image recognition and trajectory tracking according to claim 1, characterized in that, When the behavior analysis unit identifies abnormal behavior based on changes in passenger position coordinates and motion state in the continuous trajectory sequence, it includes: The behavior analysis unit derives the passenger's instantaneous motion trend parameters from the continuous trajectory sequence. The instantaneous motion trend parameters include instantaneous velocity magnitude, acceleration magnitude, and rate of change of motion direction. The instantaneous motion trend parameters are compared with a preset behavioral safety threshold range. If the instantaneous motion trend parameters exceed the behavioral safety threshold range, the current passenger is determined to be exhibiting abnormal violent movement behavior. The distribution density and dwell time of the passenger's position coordinates in the carriage space are analyzed. If the passenger's dwell time in the non-stop area exceeds a preset time limit or the position coordinates show a sudden change that matches the characteristics of a fall, the current passenger is determined to be exhibiting abnormal behavior of lingering or falling.

10. The passenger behavior analysis system based on image recognition and trajectory tracking according to claim 1, characterized in that, When the image acquisition unit acquires continuous time-series carriage image data, it includes: The image acquisition unit captures visual information inside the carriage; it adds a timestamp tag to each frame of visual information to establish a mapping relationship between the visual information and the continuous time series; it stores the visual information with the timestamp tag into a data cache queue and outputs the carriage image data of the continuous time series in chronological order.

Images