A public transportation safety early warning method and system based on real-time platform data fusion

Abstract

The present invention discloses a bus safety early warning method and system based on platform real-time data fusion. Firstly, vehicle-mounted acquisition equipment and platform acquisition equipment are used to collect data and obtain collected data. Then, data preprocessing is performed on the collected data to obtain preprocessed data. Then, data fusion is performed on the preprocessed data to obtain fused data. The present invention realizes the function of forming multi-dimensional safety early warning indicators by collecting multi-dimensional information including passenger flow, road conditions, video monitoring and vehicle operation and performing data preprocessing and data fusion. The constructed safety early warning model can be used to predict possible safety hazards of buses. Comprehensive monitoring and accurate early warning of bus safety are also realized. The method has the characteristics of high automation and intelligence, improves the accuracy and efficiency of bus safety early warning, can effectively predict and respond to potential safety hazards, and ensure the safety of passengers and drivers. The method is suitable for being widely promoted and used.

Description

Technical Field

[0001] The present invention relates to the field of intelligent transportation technology, and in particular to a public transportation safety early warning method and system based on real-time platform data fusion. Background Art

[0002] With the acceleration of urbanization, public transportation plays an increasingly important role in people's daily lives. As an important part of urban public transportation, the safety of the bus system has attracted increasing attention.

[0003] At present, the existing public transportation safety warning methods generally rely on manual monitoring and reporting, which leads to slow bus response and low warning accuracy in the process of public transportation safety warning, which makes the public transportation system have certain safety hazards during operation; therefore, it is necessary to design a public transportation safety warning method and system based on real-time platform data fusion. Summary of the invention

[0004] The purpose of the present invention is to overcome the shortcomings of the prior art, in order to better and effectively solve the problem that the existing public transportation safety warning methods generally rely on manual monitoring and reporting, resulting in slow public transportation response speed and low warning accuracy in the public transportation safety warning process, a public transportation safety warning method and system based on platform real-time data fusion is provided, which realizes the function of collecting multi-dimensional information including passenger flow, road conditions, video monitoring and vehicle operation and performing data preprocessing and data fusion to form multi-dimensional safety warning indicators, and the constructed safety warning model can be used to predict the potential safety hazards of public transportation vehicles, and also realizes comprehensive monitoring and accurate warning of public transportation vehicle safety, has the characteristics of high automation and intelligence, improves the accuracy and efficiency of public transportation safety warning, and can effectively predict and respond to potential safety hazards, thereby ensuring the safety of passengers and drivers.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is:

[0006] A public transportation safety early warning method based on platform real-time data fusion includes the following steps:

[0007] Step A, using vehicle-mounted collection equipment and platform collection equipment to collect data and obtain collected data;

[0008] Step B, performing data preprocessing on the collected data to obtain preprocessed data;

[0009] Step C, performing data fusion on the preprocessed data to obtain fused data;

[0010] Step D: construct a public transportation safety warning model based on the fused data, and then use the public transportation safety warning model to warn the safety of the bus and obtain public transportation safety warning information;

[0011] Step E: Release the public transportation safety warning information to complete the public transportation safety warning operation.

[0012] The aforementioned method for public transportation safety warning based on platform real-time data fusion, step A, uses vehicle-mounted collection equipment and platform collection equipment to collect data to obtain collected data, wherein the collected data includes platform passenger flow information, road condition information, video surveillance information and vehicle operation information, the platform passenger flow information includes boarding passenger flow, alighting passenger flow and platform waiting passenger flow data, the road condition information includes road congestion, road traffic flow, road surface conditions and construction conditions, the video surveillance information includes platform monitoring video and vehicle internal monitoring video, and the vehicle operation information includes vehicle number, license plate number and vehicle status.

[0013] The aforementioned bus safety warning method based on platform real-time data fusion, step B, performs data preprocessing on the collected data to obtain preprocessed data, wherein the specific content of the data preprocessing includes data cleaning, formatting and normalization processing, and the specific steps are as follows:

[0014] Step B1, clean the collected data to obtain cleaned data. The specific steps are as follows:

[0015] Step B11, backing up the platform passenger flow information, road condition information, video surveillance information and vehicle operation information in the collected data;

[0016] Step B12, identifying missing values ​​for each data field of the platform passenger flow information, road condition information, video surveillance information, and vehicle operation information in the collected data. If a missing value is found in a data field, the entire data record containing the missing value is deleted, wherein the missing value includes null value, NaN, and None;

[0017] Step B13, performing outlier analysis on each data field of the platform passenger flow information, road condition information, video surveillance information, and vehicle operation information in the collected data, and then setting a threshold. If the value of a data field exceeds the threshold, it is regarded as an outlier, and the entire data record containing the outlier is deleted;

[0018] Step B14, performing duplicate value detection on the platform passenger flow information, road condition information, video surveillance information and vehicle operation information in the collected data, and if duplicate data is detected, deleting the redundant data;

[0019] Step B2, formatting the cleaned data to obtain formatted data, the specific steps are as follows:

[0020] Step B21, formatting the platform passenger flow information, road condition information, video surveillance information and vehicle operation information in the cleaned data, specifically converting all timestamp data into a standard format;

[0021] Step B22, detecting the data in the field format of platform passenger flow information, road condition information, video surveillance information and vehicle operation information in the cleaned data, and removing non-numeric characters if it is detected that the data in the field format of the data contains non-numeric characters, wherein the non-numeric characters include commas, currency symbols and spaces;

[0022] Step B23, detecting the data in the cleaned data whose field format is text, including the platform passenger flow information, road condition information, video surveillance information, and vehicle operation information. If it is detected that the data in the field format is in uppercase, case conversion is performed and all uppercase is changed to lowercase, and spaces, special characters, and extra punctuation marks in the data in the field format are removed;

[0023] Step B24, detecting the location data of the platform passenger flow information, road condition information, video surveillance information and vehicle operation information in the cleaned data, and if the location data exists in different coordinate systems, converting the location data into a unified coordinate system;

[0024] Step B3, normalize the formatted data to obtain pre-processed data. The specific steps are as follows:

[0025] Step B31, using the minimum-maximum normalization method to normalize the inbound and outbound passenger flow data in the platform passenger flow data in the formatted data;

[0026] Step B32, normalizing the traffic condition data in the formatted data by using the minimum-maximum normalization method to normalize the congestion index data and the road condition score data in the traffic condition data;

[0027] Step B33, normalizing the vehicle speed data and fuel consumption data in the vehicle operation data in the formatted data by using the minimum-maximum normalization method.

[0028] The aforementioned bus safety warning method based on platform real-time data fusion, step C, performs data fusion on the pre-processed data to obtain fused data, wherein the specific content of data fusion includes time-space alignment, feature extraction and Kalman filter fusion, and the specific steps are as follows:

[0029] Step C1, performing time-space alignment on the pre-processed data, wherein the time-space alignment specifically uses timestamp synchronization technology and geographic coordinate conversion methods to accurately align platform passenger flow data, road condition data, video surveillance data and vehicle operation data to milliseconds in time and centimeters in space. The specific steps are as follows:

[0030] Step C11, collecting data points of platform passenger flow information, road condition information, video surveillance information and vehicle operation information in the pre-processed data, and extracting a timestamp associated with each data point;

[0031] Step C12, using the time synchronization protocol NTP to synchronize the timestamps of the platform passenger flow information, road condition information, video surveillance information and vehicle operation information data with the server time;

[0032] Step C13, sorting the platform passenger flow information, road condition information, video surveillance information and vehicle operation information data according to timestamps, and directly selecting the data value of the known timestamp closest to the missing timestamp as the estimated value for the data with incomplete timestamp matching, and then aligning the timestamps of all data sources;

[0033] Step C2, feature extraction is performed on the pre-processed data, wherein the feature extraction specifically uses convolutional neural network (CNN) and image processing technology to identify abnormal behaviors and congestion from video surveillance data and combines the speed and acceleration parameters in the vehicle operation data to extract features reflecting the vehicle status, and then extracts features reflecting passenger flow dynamics and road conditions through passenger flow statistics and road condition analysis. The specific steps are as follows:

[0034] Step C21, identifying abnormal behaviors and congestion in the video surveillance information data in the preprocessed data, extracting continuous video frames and selecting the VGG model for image classification, and then annotating the video frames and marking the areas of abnormal behaviors and congestion;

[0035] Step C22, extracting features from the speed and acceleration parameters in the vehicle running data in the preprocessed data, and using the extracted vehicle speed and vehicle acceleration as features;

[0036] Step C23, extracting features from the platform passenger flow information and the passenger flow statistics and road condition analysis in the road condition information in the pre-processed data, and then using the inter-frame difference method to count the passenger flow of the video surveillance data in the pre-processed data, and then analyzing the distribution of passenger flow at different times and in different areas, and then extracting features reflecting the dynamics of passenger flow;

[0037] Step C3, Kalman filter fusion is performed on the pre-processed data, wherein the Kalman filter fusion specifically performs weighted processing on vehicle motion characteristics, driver behavior characteristics, environmental perception characteristics, vehicle state characteristics and time-space characteristics, and then predicts and updates the characteristics of each time step according to the recursive formula of the Kalman filter to form a safety warning index, the safety warning index includes a vehicle abnormal state index, a road congestion index, and a passenger flow density index. The specific steps are as follows:

[0038] Step C31, sorting the preprocessed data according to the timestamp, and setting the first data as the initial position and speed estimation value of the vehicle, and setting the initial covariance matrix;

[0039] Step C32, predicting the state and covariance matrix of the next time step according to the state transfer equation of the Kalman filter and based on the initial position, speed estimate and initial covariance matrix of the vehicle;

[0040] Step C33, using the recursive formula of the Kalman filter for each time step to predict and update the state quantities of the vehicle motion characteristics, driver behavior characteristics, environmental perception characteristics, vehicle state characteristics, and time-space characteristics, and assigning different weights to each observation quantity according to the difference in the observation quantities of the vehicle motion characteristics, driver behavior characteristics, environmental perception characteristics, vehicle state characteristics, and time-space characteristics during the updating process;

[0041] Step C34, calculating the safety warning index based on the state estimation value of the vehicle motion characteristics, driver behavior characteristics, environmental perception characteristics, vehicle state characteristics and time-space characteristics after fusion of Kalman filtering.

[0042] The aforementioned bus safety warning method based on platform real-time data fusion, step D, builds a bus safety warning model based on the fused data, and then uses the bus safety warning model to warn the safety of the bus to obtain bus safety warning information. The specific steps are as follows:

[0043] Step D1, data preparation, specifically, dividing the platform passenger flow data, road condition data, video surveillance data and vehicle operation data in the fused data into a training set, a validation set and a test set. The specific steps are as follows:

[0044] Step D11, randomly extracting 70% of the data sources of the platform passenger flow data, road condition data, video surveillance data and vehicle operation data in the fused data as training sets;

[0045] Step D12, randomly extract 75% of the data sources from the data after the fusion data training set is extracted as the validation set, and the remaining 25% of the data sources are used as the test set;

[0046] Step D2, feature selection and extraction, specifically, selecting vehicle motion features, driver behavior features, environmental perception features, vehicle state features and time-space features from the training set, and then converting and reducing the dimensions of the vehicle motion features, driver behavior features, environmental perception features, vehicle state features and time-space features. The specific steps are as follows:

[0047] Step D21, collecting feature data of all vehicle motion data, driver behavior data, environmental perception data, vehicle status data, and spatiotemporal data from the training set and forming a new data set, and then dividing the data set into feature and target variables;

[0048] Step D22, based on the collected feature data of all vehicle motion data, driver behavior data, environmental perception data, vehicle status data, and spatiotemporal data, scikit-learn is used as the machine learning library, and the GradientBoosting Classifier is used to set the gradient boosting machine model, and then the prepared features and target variables are used to train the gradient boosting machine model, and then the features in the data set are sorted according to the frequency of model use, and the top-ranked features are selected to form a feature subset;

[0049] Step D23, using the validation set data to evaluate the performance of the set gradient boosting machine model and repeating the operation to find the best feature subset;

[0050] Step D3, model selection and training, specifically, based on the extracted feature data of vehicle motion characteristics, driver behavior characteristics, environmental perception characteristics, vehicle state characteristics, spatiotemporal characteristics and the complexity of public transportation safety issues, the convolutional neural network CNN model in the deep learning model is selected as the basis of the safety warning model, and then the training set is used to train the safety warning model and adjust the parameters and structure of the safety warning model. The specific steps are as follows:

[0051] Step D31, constructing a network structure of a convolutional neural network (CNN) model according to the characteristics of the security warning task, wherein the network structure of the convolutional neural network (CNN) model includes a convolutional layer, a pooling layer, an activation function, a fully connected layer, and an output layer;

[0052] Step D32, using the training set data to train the constructed convolutional neural network (CNN) model, wherein the parameters of the model need to be adjusted during the training process, and the parameters of the adjusted model include convolution kernel weights and bias terms;

[0053] Step D33, adjusting and optimizing the structure and parameters of the constructed convolutional neural network (CNN) model according to the performance during the training process, and the adjustment and optimization specifically includes modifying the number of network layers and adjusting the learning rate;

[0054] Step D4, model verification and tuning, specifically, using the verification set to evaluate the performance of the trained safety warning model, and then calculating the accuracy, recall rate and F1 value indicators to measure the effect of the safety warning model in identifying safety hazards and obtain verification results, and then adjust and optimize the model according to the verification results;

[0055] Step D5, model testing and deployment, specifically, the optimized model is finally tested on the test set. If the model test passes, the model is deployed to the vehicle safety warning unit and receives the fused data in real time. The model is then used to predict and warn of safety hazards and obtain public transportation safety warning information. The specific steps are as follows:

[0056] Step D51, use the test set data to perform a final test on the optimized convolutional neural network CNN model, and then evaluate the performance of the convolutional neural network CNN model on the test set. The evaluation content includes accuracy, recall rate, and F1 score. If the performance of the constructed convolutional neural network CNN model on the test set meets the expected requirements of public transportation safety warning, the model test passes, otherwise return to step D4;

[0057] Step D52, deploy the tested convolutional neural network CNN model into the public transportation vehicle safety warning system, then input the preprocessed data into the deployed convolutional neural network CNN model, and then use the convolutional neural network CNN model to predict safety hazards and generate corresponding public transportation safety warning information.

[0058] The aforementioned bus safety warning method based on platform real-time data fusion, step E, publishes the bus safety warning information to complete the bus safety warning operation, wherein the publishing method includes platform display screen publishing, voice broadcast publishing and sending warning information to the bus company. The specific steps are as follows:

[0059] Step E1, publishing the public transportation safety warning information on the platform display screen, wherein the public transportation safety warning information publishing unit is connected to the display screen system of the bus station, and if a potential safety hazard is detected, a corresponding warning information will be immediately generated and displayed to passengers through the display screen, and the corresponding warning information includes text prompts, icon warnings or animation demonstrations;

[0060] Step E2, the public transportation safety warning information is released by voice broadcast, wherein the release unit of the public transportation safety warning information can also convey the warning information to passengers and bus drivers through the voice broadcast unit. If a potential safety hazard is detected, a warning voice message will be generated and the warning voice message will be used to inform relevant personnel of precautions and countermeasures by voice;

[0061] Step E3, sending the bus safety warning information to the bus company, wherein the bus safety warning information issuing unit can send the warning information to the bus company, and the bus company can take corresponding management measures according to the received warning information, and the corresponding management measures include adjusting the bus number and strengthening safety inspections.

[0062] A public transportation safety warning system based on platform real-time data fusion includes a data acquisition module, a preprocessing module, a data fusion module, a public transportation safety warning module and a warning information release module, wherein the data acquisition module is used to use vehicle-mounted acquisition equipment and platform acquisition equipment to perform data acquisition and obtain the collected data; the preprocessing module is used to perform data preprocessing on the collected data and obtain preprocessed data; the data fusion module is used to perform data fusion on the preprocessed data and obtain fused data; the public transportation safety warning module is used to construct a public transportation safety warning model based on the fused data, use the public transportation safety warning model to warn the safety of the bus and obtain public transportation safety warning information; the warning information release module is used to release the public transportation safety warning information and complete the public transportation safety warning operation.

[0063] The aforementioned public transportation safety warning system based on platform real-time data fusion, the data collected in the data collection module specifically collects content including platform passenger flow information, road condition information, video surveillance information and vehicle operation information, the platform passenger flow information includes boarding passenger flow, alighting passenger flow and platform waiting passenger flow data, the road condition information includes road congestion, road traffic flow, road surface conditions and construction conditions, the video surveillance information includes platform monitoring video and vehicle internal monitoring video, and the vehicle operation information includes vehicle number, license plate number and vehicle status.

[0064] In the aforementioned public transportation safety warning system based on real-time platform data fusion, the data preprocessing in the preprocessing module specifically includes data cleaning, formatting and normalization processing.

[0065] The aforementioned public transportation safety warning system based on platform real-time data fusion, the data fusion of the data fusion module specifically includes time-space alignment, feature extraction and Kalman filter fusion.

[0066] The beneficial effects of the present invention are:

[0067] (1) First, the vehicle-mounted data collection equipment and the platform data collection equipment are used to collect data and obtain the collected data, then the collected data is preprocessed to obtain the preprocessed data, then the preprocessed data is fused to obtain the fused data, and then a public transportation safety warning model is constructed based on the fused data, and then the public transportation safety warning model is used to warn the safety of the bus and obtain the public transportation safety warning information, and then the public transportation safety warning information is released and the public transportation safety warning operation is completed; the present invention effectively realizes the function of forming a multi-dimensional safety warning indicator by collecting multi-dimensional information including passenger flow, road conditions, video surveillance and vehicle operation and performing data preprocessing and data fusion, and the safety warning model constructed based on the fused data can be used to predict the potential safety hazards of public transportation vehicles.

[0068] (2) The present invention includes data collection, preprocessing, fusion, warning model construction and warning information release modules. In this way, the bus stop data can be analyzed in real time and warning information can be released by using a variety of sensor devices and information release methods, thereby improving the safety of public buses, and can effectively predict and respond to potential safety hazards, thereby ensuring the safety of passengers and drivers.

[0069] (3) The present invention realizes comprehensive monitoring and accurate early warning of public transportation vehicle safety by deepening data collection, optimizing data preprocessing, innovating data fusion, upgrading safety early warning model construction and diversifying early warning information release. It has the characteristics of high automation and intelligence, and improves the accuracy and efficiency of public transportation safety early warning. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 It is an overall flow chart of a public transportation safety early warning method based on platform real-time data fusion of the present invention;

[0071] Figure 2 is a comparison chart of early warning accuracy in an embodiment of the present invention;

[0072] Figure 3 is a comparison chart of emergency response time in an embodiment of the present invention;

[0073] Figure 4 2 is a comparison chart of the safety accident incidence rates in the embodiments of the present invention. DETAILED DESCRIPTION

[0074] The present invention will be further described below in conjunction with the accompanying drawings.

[0075] like Figure 1 As shown, a public transportation safety early warning method based on platform real-time data fusion of the present invention comprises the following steps:

[0076] Step A, using vehicle-mounted collection equipment and platform collection equipment to collect data to obtain collected data, wherein the collected data includes platform passenger flow information, road condition information, video surveillance information and vehicle operation information, the platform passenger flow information includes boarding passenger flow, alighting passenger flow and platform waiting passenger flow data, the road condition information includes road congestion, road traffic flow, road surface conditions and construction conditions, the video surveillance information includes platform monitoring video and vehicle internal monitoring video, and the vehicle operation information includes vehicle number, license plate number and vehicle status.

[0077] Among them, by installing various sensors and cameras at bus stops and in buses, real-time data including passenger flow, weather conditions and traffic conditions are collected; infrared sensors are used to monitor the number of passengers getting on and off the bus in real time, thereby obtaining passenger flow data; meteorological sensors are used to obtain real-time temperature, humidity, wind speed and wind direction weather information; high-definition cameras are used to capture traffic conditions around the platform, including vehicle flow and pedestrian flow; vehicle positioning is carried out through the GPS and Beidou dual-system composite module to obtain the vehicle's real-time position, speed and direction; and the on-board terminal is used to collect real-time information on passenger flow, driver behavior and vehicle status.

[0078] Step B, preprocessing the collected data to obtain preprocessed data, wherein the specific content of the data preprocessing includes data cleaning, formatting and normalization. The specific steps are as follows:

[0079] Step B1, clean the collected data to obtain cleaned data. The specific steps are as follows:

[0080] Step B11, backing up the platform passenger flow information, road condition information, video surveillance information and vehicle operation information in the collected data;

[0081] Step B12, identifying missing values ​​for each data field of the platform passenger flow information, road condition information, video surveillance information, and vehicle operation information in the collected data. If a missing value is found in a data field, the entire data record containing the missing value is deleted, wherein the missing value includes null value, NaN, and None;

[0082] Step B13, performing outlier analysis on each data field of the platform passenger flow information, road condition information, video surveillance information, and vehicle operation information in the collected data, and then setting a threshold. If the value of a data field exceeds the threshold, it is regarded as an outlier, and the entire data record containing the outlier is deleted;

[0083] Step B14, performing duplicate value detection on the platform passenger flow information, road condition information, video surveillance information and vehicle operation information in the collected data, and if duplicate data is detected, deleting the redundant data;

[0084] Step B2, formatting the cleaned data to obtain formatted data, the specific steps are as follows:

[0085] Step B21, formatting the platform passenger flow information, road condition information, video surveillance information and vehicle operation information in the cleaned data, specifically converting all timestamp data into a standard format;

[0086] Step B22, detecting the data in the field format of platform passenger flow information, road condition information, video surveillance information and vehicle operation information in the cleaned data, and removing non-numeric characters if it is detected that the data in the field format of the data contains non-numeric characters, wherein the non-numeric characters include commas, currency symbols and spaces;

[0087] Step B23, detecting the data in the cleaned data whose field format is text, including the platform passenger flow information, road condition information, video surveillance information, and vehicle operation information. If it is detected that the data in the field format is in uppercase, case conversion is performed and all uppercase is changed to lowercase, and spaces, special characters, and extra punctuation marks in the data in the field format are removed;

[0088] Step B24, detecting the location data of the platform passenger flow information, road condition information, video surveillance information and vehicle operation information in the cleaned data, and if the location data exists in different coordinate systems, converting the location data into a unified coordinate system;

[0089] Step B3, normalize the formatted data to obtain pre-processed data. The specific steps are as follows:

[0090] Step B31, using the minimum-maximum normalization method to normalize the inbound and outbound passenger flow data in the platform passenger flow data in the formatted data;

[0091] Step B32, normalizing the traffic condition data in the formatted data by using the minimum-maximum normalization method to normalize the congestion index data and the road condition score data in the traffic condition data;

[0092] Step B33, normalizing the vehicle speed data and fuel consumption data in the vehicle operation data in the formatted data by using the minimum-maximum normalization method.

[0093] Step C, performing data fusion on the preprocessed data to obtain fused data, wherein the specific contents of data fusion include time-space alignment, feature extraction and Kalman filter fusion. The specific steps are as follows:

[0094] Step C1, performing time-space alignment on the pre-processed data, wherein the time-space alignment specifically uses timestamp synchronization technology and geographic coordinate conversion methods to accurately align platform passenger flow data, road condition data, video surveillance data and vehicle operation data to milliseconds in time and centimeters in space. The specific steps are as follows:

[0095] Step C11, collecting data points of platform passenger flow information, road condition information, video surveillance information and vehicle operation information in the pre-processed data, and extracting a timestamp associated with each data point;

[0096] Step C12, using the time synchronization protocol NTP to synchronize the timestamps of the platform passenger flow information, road condition information, video surveillance information and vehicle operation information data with the server time;

[0097] Step C13, sorting the platform passenger flow information, road condition information, video surveillance information and vehicle operation information data according to timestamps, and directly selecting the data value of the known timestamp closest to the missing timestamp as the estimated value for the data with incomplete timestamp matching, and then aligning the timestamps of all data sources;

[0098] Step C2, feature extraction is performed on the pre-processed data, wherein the feature extraction specifically uses convolutional neural network (CNN) and image processing technology to identify abnormal behaviors and congestion from video surveillance data and combines the speed and acceleration parameters in the vehicle operation data to extract features reflecting the vehicle status, and then extracts features reflecting passenger flow dynamics and road conditions through passenger flow statistics and road condition analysis. The specific steps are as follows:

[0099] Step C21, identifying abnormal behaviors and congestion in the video surveillance information data in the preprocessed data, extracting continuous video frames and selecting the VGG model for image classification, and then annotating the video frames and marking the areas of abnormal behaviors and congestion;

[0100] Step C22, extracting features from the speed and acceleration parameters in the vehicle running data in the preprocessed data, and using the extracted vehicle speed and vehicle acceleration as features;

[0101] Step C23, extracting features from the platform passenger flow information and the passenger flow statistics and road condition analysis in the road condition information in the pre-processed data, and then using the inter-frame difference method to count the passenger flow of the video surveillance data in the pre-processed data, and then analyzing the distribution of passenger flow at different times and in different areas, and then extracting features reflecting the dynamics of passenger flow;

[0102] Step C3, Kalman filter fusion is performed on the pre-processed data, wherein the Kalman filter fusion specifically performs weighted processing on vehicle motion characteristics, driver behavior characteristics, environmental perception characteristics, vehicle state characteristics and time-space characteristics, and then predicts and updates the characteristics of each time step according to the recursive formula of the Kalman filter to form a safety warning index, the safety warning index includes a vehicle abnormal state index, a road congestion index, and a passenger flow density index. The specific steps are as follows:

[0103] Step C31, sorting the preprocessed data according to the timestamp, and setting the first data as the initial position and speed estimation value of the vehicle, and setting the initial covariance matrix;

[0104] Step C32, predicting the state and covariance matrix of the next time step according to the state transfer equation of the Kalman filter and based on the initial position, speed estimate and initial covariance matrix of the vehicle;

[0105] Step C33, using the recursive formula of the Kalman filter for each time step to predict and update the state quantities of the vehicle motion characteristics, driver behavior characteristics, environmental perception characteristics, vehicle state characteristics, and time-space characteristics, and assigning different weights to each observation quantity according to the difference in the observation quantities of the vehicle motion characteristics, driver behavior characteristics, environmental perception characteristics, vehicle state characteristics, and time-space characteristics during the updating process;

[0106] Step C34, calculating the safety warning index based on the state estimation value of the vehicle motion characteristics, driver behavior characteristics, environmental perception characteristics, vehicle state characteristics and time-space characteristics after fusion of Kalman filtering.

[0107] Step D: construct a public transportation safety warning model based on the fused data, and then use the public transportation safety warning model to warn the safety of the bus and obtain public transportation safety warning information. The specific steps are as follows:

[0108] Step D1, data preparation, specifically, dividing the platform passenger flow data, road condition data, video surveillance data and vehicle operation data in the fused data into a training set, a validation set and a test set. The specific steps are as follows:

[0109] Step D11, randomly extracting 70% of the data sources of the platform passenger flow data, road condition data, video surveillance data and vehicle operation data in the fused data as training sets;

[0110] Step D12, randomly extract 75% of the data sources from the data after the fusion data training set is extracted as the validation set, and the remaining 25% of the data sources are used as the test set;

[0111] Step D2, feature selection and extraction, specifically, selecting vehicle motion features, driver behavior features, environmental perception features, vehicle state features and time-space features from the training set, and then converting and reducing the dimensions of the vehicle motion features, driver behavior features, environmental perception features, vehicle state features and time-space features. The specific steps are as follows:

[0112] Step D21, collecting feature data of all vehicle motion data, driver behavior data, environmental perception data, vehicle status data, and spatiotemporal data from the training set and forming a new data set, and then dividing the data set into feature and target variables;

[0113] Step D22, based on the collected feature data of all vehicle motion data, driver behavior data, environmental perception data, vehicle status data, and spatiotemporal data, scikit-learn is used as the machine learning library, and the GradientBoosting Classifier is used to set the gradient boosting machine model, and then the prepared features and target variables are used to train the gradient boosting machine model, and then the features in the data set are sorted according to the frequency of model use, and the top-ranked features are selected to form a feature subset;

[0114] Step D23, using the validation set data to evaluate the performance of the set gradient boosting machine model and repeating the operation to find the best feature subset;

[0115] Step D3, model selection and training, specifically, based on the extracted feature data of vehicle motion characteristics, driver behavior characteristics, environmental perception characteristics, vehicle state characteristics, spatiotemporal characteristics and the complexity of public transportation safety issues, the convolutional neural network CNN model in the deep learning model is selected as the basis of the safety warning model, and then the training set is used to train the safety warning model and adjust the parameters and structure of the safety warning model. The specific steps are as follows:

[0116] Step D31, constructing a network structure of a convolutional neural network (CNN) model according to the characteristics of the security warning task, wherein the network structure of the convolutional neural network (CNN) model includes a convolutional layer, a pooling layer, an activation function, a fully connected layer, and an output layer;

[0117] Step D32, using the training set data to train the constructed convolutional neural network (CNN) model, wherein the parameters of the model need to be adjusted during the training process, and the parameters of the adjusted model include convolution kernel weights and bias terms;

[0118] Step D33, adjusting and optimizing the structure and parameters of the constructed convolutional neural network (CNN) model according to the performance during the training process, and the adjustment and optimization specifically includes modifying the number of network layers and adjusting the learning rate;

[0119] Step D4, model verification and tuning, specifically, using the verification set to evaluate the performance of the trained safety warning model, and then calculating the accuracy, recall rate and F1 value indicators to measure the effect of the safety warning model in identifying safety hazards and obtain verification results, and then adjust and optimize the model according to the verification results;

[0120] Step D5, model testing and deployment, specifically, the optimized model is finally tested on the test set. If the model test passes, the model is deployed to the vehicle safety warning unit and receives the fused data in real time. The model is then used to predict and warn of safety hazards and obtain public transportation safety warning information. The specific steps are as follows:

[0121] Step D51, use the test set data to perform a final test on the optimized convolutional neural network CNN model, and then evaluate the performance of the convolutional neural network CNN model on the test set. The evaluation content includes accuracy, recall rate, and F1 score. If the performance of the constructed convolutional neural network CNN model on the test set meets the expected requirements of public transportation safety warning, the model test passes, otherwise return to step D4;

[0122] Step D52, deploy the tested convolutional neural network CNN model into the public transportation vehicle safety warning system, then input the preprocessed data into the deployed convolutional neural network CNN model, and then use the convolutional neural network CNN model to predict safety hazards and generate corresponding public transportation safety warning information.

[0123] Step E: Release the public transportation safety warning information to complete the public transportation safety warning operation. The release methods include platform display screen release, voice broadcast release and sending warning information to the bus company. The specific steps are as follows:

[0124] Step E1, publishing the public transportation safety warning information on the platform display screen, wherein the public transportation safety warning information publishing unit is connected to the display screen system of the bus station, and if a potential safety hazard is detected, a corresponding warning information will be immediately generated and displayed to passengers through the display screen, and the corresponding warning information includes text prompts, icon warnings or animation demonstrations;

[0125] Step E2, the public transportation safety warning information is released by voice broadcast, wherein the release unit of the public transportation safety warning information can also convey the warning information to passengers and bus drivers through the voice broadcast unit. If a potential safety hazard is detected, a warning voice message will be generated and the warning voice message will be used to inform relevant personnel of precautions and countermeasures by voice;

[0126] Step E3, sending the bus safety warning information to the bus company, wherein the bus safety warning information issuing unit can send the warning information to the bus company, and the bus company can take corresponding management measures according to the received warning information, and the corresponding management measures include adjusting the bus number and strengthening safety inspections.

[0127] A public transportation safety warning system based on platform real-time data fusion includes a data acquisition module, a preprocessing module, a data fusion module, a public transportation safety warning module and a warning information release module, wherein the data acquisition module is used to use vehicle-mounted acquisition equipment and platform acquisition equipment to perform data acquisition and obtain the collected data; the preprocessing module is used to perform data preprocessing on the collected data and obtain preprocessed data; the data fusion module is used to perform data fusion on the preprocessed data and obtain fused data; the public transportation safety warning module is used to construct a public transportation safety warning model based on the fused data, use the public transportation safety warning model to warn the safety of the bus and obtain public transportation safety warning information; the warning information release module is used to release the public transportation safety warning information and complete the public transportation safety warning operation.

[0128] Specifically, the collected data in the data collection module include platform passenger flow information, road condition information, video surveillance information and vehicle operation information. The platform passenger flow information includes boarding passenger flow, alighting passenger flow and platform waiting passenger flow data. The road condition information includes road congestion, road traffic flow, road surface conditions and construction conditions. The video surveillance information includes platform monitoring video and vehicle internal monitoring video. The vehicle operation information includes vehicle number, license plate number and vehicle status.

[0129] Specifically, the data preprocessing in the preprocessing module includes data cleaning, formatting and normalization.

[0130] Specifically, the data fusion of the data fusion module includes time-space alignment, feature extraction and Kalman filter fusion.

[0131] In order to better illustrate the use effect of the present invention, a specific embodiment of the method of the present invention is introduced below.

[0132] This embodiment uses the public transportation safety early warning method and system of the present invention to conduct a one-month experimental comparison on a busy public transportation line in the city. The index comparison data is shown in Table 1.

[0133] Table 1. Comparison of indicators

[0134]

[0135] In summary, the public transportation safety warning method and system based on platform real-time data fusion of the present invention realizes the function of forming multi-dimensional safety warning indicators by collecting multi-dimensional information including passenger flow, road conditions, video surveillance and vehicle operation and performing data preprocessing and data fusion, and the constructed safety warning model can be used to predict the potential safety hazards of public transportation vehicles, and also realizes comprehensive monitoring and accurate warning of public transportation vehicle safety, has the characteristics of high degree of automation and intelligence, improves the accuracy and efficiency of public transportation safety warning, and can effectively predict and respond to potential safety hazards, thereby ensuring the safety of passengers and drivers.

[0136] The above shows and describes the basic principles, main features and advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A public transportation safety early warning method based on real-time platform data fusion, characterized by: The following steps are included: Step A, using vehicle-mounted data collection equipment and platform data collection equipment to collect data and obtain collected data, wherein the collected data includes platform passenger flow information, road condition information, video surveillance information and vehicle operation information, the platform passenger flow information includes boarding passenger flow, alighting passenger flow and platform waiting passenger flow data, the road condition information includes road congestion, road traffic flow, road surface conditions and construction conditions, the video surveillance information includes platform surveillance video and vehicle internal surveillance video, and the vehicle operation information includes vehicle number, license plate number and vehicle status; Step B, preprocessing the collected data to obtain preprocessed data, wherein the specific content of the data preprocessing includes data cleaning, formatting and normalization. The specific steps are as follows: Step B1, clean the collected data to obtain cleaned data. The specific steps are as follows: Step B11, backing up the platform passenger flow information, road condition information, video surveillance information and vehicle operation information in the collected data; Step B12, identifying missing values ​​for each data field of the platform passenger flow information, road condition information, video surveillance information, and vehicle operation information in the collected data. If a missing value is found in a data field, the entire data record containing the missing value is deleted, wherein the missing value includes null value, NaN, and None; Step B13, performing outlier analysis on each data field of the platform passenger flow information, road condition information, video surveillance information, and vehicle operation information in the collected data, and then setting a threshold. If the value of a data field exceeds the threshold, it is considered an outlier, and the entire data record containing the outlier is deleted; Step B14, performing duplicate value detection on the platform passenger flow information, road condition information, video surveillance information and vehicle operation information in the collected data, and if duplicate data is detected, deleting the redundant data; Step B2, formatting the cleaned data to obtain formatted data, the specific steps are as follows: Step B21, formatting the platform passenger flow information, road condition information, video surveillance information and vehicle operation information in the cleaned data, specifically converting all timestamp data into a standard format; Step B22, detecting the data in the field format of platform passenger flow information, road condition information, video surveillance information and vehicle operation information in the cleaned data, and removing non-numeric characters if it is detected that the data in the field format of the data contains non-numeric characters, wherein the non-numeric characters include commas, currency symbols and spaces; Step B23, detecting the data in the cleaned data whose field format is text, including the platform passenger flow information, road condition information, video surveillance information, and vehicle operation information. If it is detected that the data in the field format is in uppercase, case conversion is performed and all uppercase is changed to lowercase, and spaces, special characters, and extra punctuation marks in the data in the field format are removed; Step B24, detecting the location data of the platform passenger flow information, road condition information, video surveillance information and vehicle operation information in the cleaned data, and if the location data exists in different coordinate systems, converting the location data into a unified coordinate system; Step B3, normalize the formatted data to obtain pre-processed data. The specific steps are as follows: Step B31, using the minimum-maximum normalization method to normalize the inbound and outbound passenger flow data in the platform passenger flow data in the formatted data; Step B32, normalizing the traffic condition data in the formatted data by using the minimum-maximum normalization method to normalize the congestion index data and the road condition score data in the traffic condition data; Step B33, normalizing the vehicle speed data and fuel consumption data in the vehicle operation data in the formatted data by using the minimum-maximum normalization method; Step C, performing data fusion on the preprocessed data to obtain fused data, wherein the specific contents of data fusion include time-space alignment, feature extraction and Kalman filter fusion. The specific steps are as follows: Step C1, performing time-space alignment on the pre-processed data, wherein the time-space alignment specifically uses timestamp synchronization technology and geographic coordinate conversion methods to accurately align platform passenger flow data, road condition data, video surveillance data and vehicle operation data to milliseconds in time and centimeters in space. The specific steps are as follows: Step C11, collecting data points of platform passenger flow information, road condition information, video surveillance information and vehicle operation information in the pre-processed data, and extracting a timestamp associated with each data point; Step C12, using the time synchronization protocol NTP to synchronize the timestamps of the platform passenger flow information, road condition information, video surveillance information and vehicle operation information data with the server time; Step C13, sorting the platform passenger flow information, road condition information, video surveillance information and vehicle operation information data according to timestamps, and directly selecting the data value of the known timestamp closest to the missing timestamp as the estimated value for the data with incomplete timestamp matching, and then aligning the timestamps of all data sources; Step C2, feature extraction is performed on the pre-processed data, wherein the feature extraction specifically uses the convolution neural network (CNN) and image processing technology to identify abnormal behaviors and congestion from the video surveillance data and combines the speed and acceleration parameters in the vehicle operation data to extract features reflecting the vehicle status, and then extracts features reflecting the passenger flow dynamics and road conditions through passenger flow statistics and road condition analysis. The specific steps are as follows: Step C21, identifying abnormal behaviors and congestion in the video surveillance information data in the preprocessed data, extracting continuous video frames and selecting the VGG model for image classification, and then annotating the video frames and marking the areas of abnormal behaviors and congestion; Step C22, extracting features from the speed and acceleration parameters in the vehicle running data in the preprocessed data, and using the extracted vehicle speed and vehicle acceleration as features; Step C23, extracting features from the platform passenger flow information and the passenger flow statistics and road condition analysis in the road condition information in the pre-processed data, and then using the inter-frame difference method to count the passenger flow of the video surveillance data in the pre-processed data, and then analyzing the distribution of passenger flow at different times and in different areas, and then extracting features reflecting the dynamics of passenger flow; Step C3, Kalman filter fusion is performed on the pre-processed data, wherein the Kalman filter fusion specifically performs weighted processing on vehicle motion characteristics, driver behavior characteristics, environmental perception characteristics, vehicle state characteristics and time-space characteristics, and then predicts and updates the characteristics of each time step according to the recursive formula of the Kalman filter to form a safety warning index, the safety warning index includes a vehicle abnormal state index, a road congestion index, and a passenger flow density index. The specific steps are as follows: Step C31, sorting the preprocessed data according to the timestamp, and setting the first data as the initial position and speed estimation value of the vehicle, and setting the initial covariance matrix; Step C32, predicting the state and covariance matrix of the next time step according to the state transfer equation of the Kalman filter and based on the initial position, speed estimate and initial covariance matrix of the vehicle; Step C33, using the recursive formula of the Kalman filter for each time step to predict and update the state quantities of the vehicle motion characteristics, driver behavior characteristics, environmental perception characteristics, vehicle state characteristics, and time-space characteristics, and assigning different weights to each observation quantity according to the difference in the observation quantities of the vehicle motion characteristics, driver behavior characteristics, environmental perception characteristics, vehicle state characteristics, and time-space characteristics during the updating process; Step C34, calculating the safety warning index based on the state estimation value of the vehicle motion characteristics, driver behavior characteristics, environmental perception characteristics, vehicle state characteristics and time-space characteristics after fusion of Kalman filtering, the specific steps are as follows: Step D1, data preparation, specifically, dividing the platform passenger flow data, road condition data, video surveillance data and vehicle operation data in the fused data into a training set, a validation set and a test set. The specific steps are as follows: Step D11, randomly extracting 70% of the data sources of the platform passenger flow data, road condition data, video surveillance data and vehicle operation data in the fused data as training sets; Step D12, randomly extract 75% of the data sources from the data after the fusion data training set is extracted as the validation set, and the remaining 25% of the data sources are used as the test set; Step D2, feature selection and extraction, specifically, selecting vehicle motion features, driver behavior features, environmental perception features, vehicle state features and time-space features from the training set, and then converting and reducing the dimensions of the vehicle motion features, driver behavior features, environmental perception features, vehicle state features and time-space features. The specific steps are as follows: Step D21, collecting feature data of all vehicle motion data, driver behavior data, environmental perception data, vehicle status data, and spatiotemporal data from the training set and forming a new data set, and then dividing the data set into feature and target variables; Step D22, based on the collected feature data of all vehicle motion data, driver behavior data, environmental perception data, vehicle status data, and spatiotemporal data, scikit-learn is used as the machine learning library, and the GradientBoosting Classifier is used to set the gradient boosting machine model, and then the prepared features and target variables are used to train the gradient boosting machine model, and then the features in the data set are sorted according to the frequency of model use, and the top-ranked features are selected to form a feature subset; Step D23, using the validation set data to evaluate the performance of the set gradient boosting machine model and repeating the operation to find the best feature subset; Step D3, model selection and training, specifically, based on the extracted feature data of vehicle motion characteristics, driver behavior characteristics, environmental perception characteristics, vehicle state characteristics, spatiotemporal characteristics and the complexity of public transportation safety issues, the convolutional neural network CNN model in the deep learning model is selected as the basis of the safety warning model, and then the training set is used to train the safety warning model and adjust the parameters and structure of the safety warning model. The specific steps are as follows: Step D31, constructing a network structure of a convolutional neural network (CNN) model according to the characteristics of the security warning task, wherein the network structure of the convolutional neural network (CNN) model includes a convolutional layer, a pooling layer, an activation function, a fully connected layer, and an output layer; Step D32, using the training set data to train the constructed convolutional neural network (CNN) model, wherein the parameters of the model need to be adjusted during the training process, and the parameters of the adjusted model include convolution kernel weights and bias terms; Step D33, adjusting and optimizing the structure and parameters of the constructed convolutional neural network (CNN) model according to the performance during the training process, and the adjustment and optimization specifically includes modifying the number of network layers and adjusting the learning rate; Step D4, model verification and tuning, specifically, using the verification set to evaluate the performance of the trained safety warning model, and then calculating the accuracy, recall rate and F1 value indicators to measure the effect of the safety warning model in identifying safety hazards and obtain verification results, and then adjust and optimize the model according to the verification results; Step D5, model testing and deployment, specifically, the optimized model is finally tested on the test set. If the model test passes, the model is deployed to the vehicle safety warning unit and receives the fused data in real time. The model is then used to predict and warn of safety hazards and obtain public transportation safety warning information. The specific steps are as follows: Step D51, use the test set data to perform a final test on the optimized convolutional neural network CNN model, and then evaluate the performance of the convolutional neural network CNN model on the test set. The evaluation content includes accuracy, recall rate, and F1 score. If the performance of the constructed convolutional neural network CNN model on the test set meets the expected requirements of public transportation safety warning, the model test passes, otherwise return to step D4; Step D52, deploying the tested convolutional neural network CNN model to the public transportation vehicle safety warning system, then inputting the preprocessed data into the deployed convolutional neural network CNN model, and then using the convolutional neural network CNN model to predict safety hazards and generate corresponding public transportation safety warning information; Step D: construct a public transportation safety warning model based on the fused data, and then use the public transportation safety warning model to warn the safety of the bus and obtain public transportation safety warning information; Step E: Release the public transportation safety warning information to complete the public transportation safety warning operation.

2. A public transportation safety early warning method based on platform real-time data fusion according to claim 1, characterized in that: Step E: Release the public transportation safety warning information to complete the public transportation safety warning operation. The release methods include platform display screen release, voice broadcast release and sending warning information to the bus company. The specific steps are as follows: Step E1, publishing the public transportation safety warning information on the platform display screen, wherein the public transportation safety warning information publishing unit is connected to the display screen system of the bus station, and if a potential safety hazard is detected, a corresponding warning information will be immediately generated and displayed to passengers through the display screen, and the corresponding warning information includes text prompts, icon warnings or animation demonstrations; Step E2, the public transportation safety warning information is released by voice broadcast, wherein the release unit of the public transportation safety warning information can also convey the warning information to passengers and bus drivers through the voice broadcast unit. If a potential safety hazard is detected, a warning voice message will be generated and the warning voice message will be used to inform relevant personnel of precautions and countermeasures by voice; Step E3, sending the bus safety warning information to the bus company, wherein the bus safety warning information issuing unit can send the warning information to the bus company, and the bus company can take corresponding management measures according to the received warning information, and the corresponding management measures include adjusting the bus number and strengthening safety inspections.

3. A public transportation safety warning system based on real-time platform data fusion, the operation process of the public transportation safety warning system is based on the public transportation safety warning method according to any one of claims 1-2, characterized in that: It includes data acquisition module, preprocessing module, data fusion module, public transportation safety warning module and warning information release module. The data acquisition module is used to collect data using vehicle-mounted acquisition equipment and platform acquisition equipment and obtain collected data; The preprocessing module is used to perform data preprocessing on the collected data and obtain preprocessed data; The data fusion module is used to perform data fusion on the pre-processed data and obtain fused data; The public transportation safety warning module is used to construct a public transportation safety warning model based on the fused data, use the public transportation safety warning model to warn the safety of the bus and obtain public transportation safety warning information; The warning information release module is used to release public transportation safety warning information and complete public transportation safety warning operations.

4. A public transportation safety warning system based on platform real-time data fusion according to claim 3, characterized in that: The specific collected data in the data collection module include platform passenger flow information, road condition information, video surveillance information and vehicle operation information. The platform passenger flow information includes boarding passenger flow, alighting passenger flow and platform waiting passenger flow data. The road condition information includes road congestion, road traffic flow, road surface conditions and construction conditions. The video surveillance information includes platform monitoring video and vehicle internal monitoring video. The vehicle operation information includes vehicle number, license plate number and vehicle status.

5. The public transportation safety warning system based on platform real-time data fusion according to claim 3 is characterized by: The data preprocessing in the preprocessing module specifically includes data cleaning, formatting and normalization.

6. The public transportation safety warning system based on platform real-time data fusion according to claim 3 is characterized by: The specific fusion contents of the data fusion of the data fusion module include time-space alignment, feature extraction and Kalman filter fusion.

Images