Platform vehicle abnormal behavior intelligent analysis method based on deep learning

Abstract

The invention discloses a platform vehicle abnormal behavior intelligent analysis method based on deep learning. Accurate early warning is realized through multi-dimensional data acquisition and deep learning model construction. Firstly, platform passenger flow, road conditions, video monitoring and vehicle operation information are integrated, and a comprehensive data basis is provided for anomaly analysis. After the data is preprocessed, a graph neural network and an attention mechanism are utilized to perform preliminary fusion, and an AGCN adaptive graph convolutional network and a DTW dynamic time warping algorithm are introduced to optimize a fusion result, so that time sequence data are effectively aligned, and the feature extraction accuracy is improved. And then, a deep learning model is constructed to automatically identify abnormal behavior modes of the platform vehicles, such as illegal parking and passenger congestion. Early warning information is issued in real time, potential risks can be timely handled, and safe and efficient operation of a public transportation system is guaranteed. According to the scheme, the identification and early warning capability of the abnormal behavior of the station vehicle is enhanced, and powerful support is provided for bus safety management.

Description

Technical Field

[0001] The present invention relates to the field of abnormal behaviors of platform vehicles, and in particular to an intelligent analysis method for abnormal behaviors of platform vehicles based on deep learning. Background Art

[0002] With the rapid development of public transportation, the increasing urban population and the growing traffic demand, the safety monitoring and abnormal analysis of bus stops and vehicles have become an important research direction. Traditional monitoring methods mainly rely on manual observation, but due to the limitations of human energy and reaction speed, it is difficult to detect and deal with potential safety issues in a timely manner. In recent years, with the rapid development of Internet of Things technology, big data analysis and deep learning technology, new opportunities have been provided for abnormal analysis of bus stops and vehicles.

[0003] At present, although some technologies have been applied to the monitoring of bus stops and vehicles, they still have deficiencies in the accuracy and intelligence of anomaly detection. Most technologies only focus on a single data source, such as vehicle location or video surveillance, and lack comprehensive analysis and in-depth mining of these data sources. In addition, traditional anomaly detection methods are often based on rules or simple statistical models, which are difficult to adapt to complex and changing traffic environments, and cannot automatically learn and identify new abnormal patterns.

[0004] In order to more effectively monitor the safety of vehicles at bus stops and promptly detect and handle abnormal situations, there is an urgent need for a solution that can comprehensively collect and analyze multi-source data and use deep learning technology for anomaly detection. This solution needs to be able to collect and analyze multi-dimensional information such as vehicle operation data, platform passenger flow data, video surveillance data, etc. in real time, and conduct in-depth mining and intelligent analysis of these data through deep learning models, so as to accurately identify potential safety issues such as abnormal vehicle behavior, platform congestion, and passenger abnormalities. Summary of the invention

[0005] This application provides a method for intelligently analyzing abnormal behaviors of vehicles on platforms based on deep learning. To solve the above problems, a deep learning model is constructed to extract features and recognize patterns from the collected multi-dimensional data, thereby realizing automatic detection and early warning of abnormal behaviors of vehicles. At the same time, real-time data collection and transmission technology is combined to ensure the timeliness and accuracy of monitoring and analysis.

[0006] This application provides a method for intelligent analysis of abnormal behavior of platform vehicles based on deep learning, including: S101, collect data using vehicle-mounted collection equipment and platform collection equipment to obtain multi-dimensional data; S102, performing data preprocessing on the collected data; S103, performing data fusion on the pre-processed data to form a multi-dimensional safety early warning indicator; S104, based on the fused data, construct a public transportation safety warning model, and use the model to issue abnormal behavior warnings; S105, releasing the warning information through various means.

[0007] Preferably, the step S101 of obtaining multi-dimensional data includes: Platform passenger flow information, road condition information, video surveillance information and vehicle operation information.

[0008] Preferably, the step S102, performing data preprocessing on the collected data, includes: Identify missing values ​​and delete entire data records containing missing values; Analyze outliers and delete the entire data record containing the outlier; Detect duplicate data and delete redundant data; Convert timestamp data to UTC time format; Remove non-numeric characters from numeric data; Convert text data to the unified UTF-8 encoding format and remove spaces; Convert the position data to the unified WGS-84 coordinate system.

[0009] Preferably, the step S104, constructing a public transportation safety early warning model, includes: Built with deep learning algorithms, it can automatically learn and identify abnormal behavior patterns of public transportation vehicles.

[0010] Preferably, the S103 includes: S201, using graph neural network to perform preliminary fusion of preprocessed data; S202, performing multi-source data fusion optimization on the fusion result of the graph neural network.

[0011] Preferably, the S102 includes: Perform feature extraction on the preprocessed data; Introducing the attention mechanism in graph neural networks; At each layer of the graph neural network or at a specific aggregation step, the attention score for each feature is calculated; Use the softmax function to convert the attention scores into weights and apply these weights to the feature aggregation process; Use labeled sample training data including normal and abnormal behaviors to train the graph neural network model; During training, the model learns how to identify abnormal behaviors based on input features and attention mechanisms.

[0012] Preferably, the S202 includes: In addition to the preliminary fusion results of the graph neural network, data from other sources, such as directly collected road condition information, original frames of video surveillance, or feature extraction results, are called new multi-source data; Time synchronization, denoising, and normalization of new multi-source data; Apply Kalman filter algorithm to initially fuse multi-source data; On the basis of Kalman filter fusion, GNN graph neural network is further introduced to perform deep fusion of multi-source data, and the forward propagation formula of GNN H(l+1)=σ(AH(l)W(l)) is used to update node features; where A is the adjacency matrix, which represents the relationship between nodes; H(l) is the node feature of the lth layer; W(l) is a learnable weight matrix; σ is an activation function, which is used to introduce nonlinear characteristics; For time series data, the original CNN convolutional neural network model is upgraded to a hybrid model combining LSTM long short-term memory network and CNN; Use labeled training data to train the entire fusion model, including GNN and hybrid model; After training is completed, new multi-source data is input into the fusion model for abnormal behavior detection.

[0013] Preferably, the S202 further includes: S301, introduce the AGCN adaptive graph convolutional network and adjust the parameters of the convolution kernel according to the actual distribution of the data; S302, applying the DTW dynamic time warping algorithm to accurately align data with different timestamps on the time axis.

[0014] Preferably, the S301 includes: Preprocess the original multi-source data; The preprocessed data is converted into a graph structure representation; the nodes represent data points, vehicles and passengers, and the edges represent the relationship between vehicle driving trajectories and passenger flow data points; Design multiple layers of graph convolutional layers, each of which contains learnable convolution kernel parameters; Assign an attention weight to each node, which is calculated based on the node's features and the features of its neighboring nodes; Design additional neural network fully connected layers to dynamically generate convolution kernel parameters based on the characteristics of the input graph; After the graph convolution layer, a graph pooling layer is introduced to reduce the number of nodes while retaining important feature information.

[0015] Preferably, the S302 includes: Preprocess the time series data of platform passenger flow information, road condition information, video surveillance information and vehicle operation information data by denoising and smoothing; Apply the DTW algorithm to accurately align these time series data on the time axis; Extract key time series features from the aligned time series data; Build an abnormal behavior recognition model based on deep learning technology, which combines CNN convolutional neural network and LSTM long short-term memory network; The features extracted by the AGCN module and the DTW module are used as input to this deep learning model for abnormal behavior recognition.

[0016] One or more technical solutions provided in this application have at least the following technical effects or advantages: 1. By collecting multi-dimensional data including platform passenger flow, road conditions, video surveillance and vehicle operation information, this technical solution provides a comprehensive insight into the operating status of the public transportation system. Data fusion technology further integrates these scattered information sources, eliminates data redundancy and contradictions, and thus generates more accurate and reliable safety warning indicators, providing a solid foundation for public transportation safety management. Automated data collection, preprocessing and fusion processes, combined with efficient early warning model calculations, ensure that the system can respond to any abnormal situation in the public transportation system in real time. This real-time nature not only improves the timeliness of early warnings, but also enables relevant departments and passengers to quickly take necessary countermeasures, effectively reducing the probability of safety accidents.

[0017] 2. The early warning model built with deep learning algorithms can automatically learn and identify abnormal behavior patterns of public transportation vehicles. This intelligent feature enables the model to continuously adapt to the dynamic changes of the public transportation system, improve the accuracy and precision of early warnings through continuous learning and optimization, and provide strong guarantees for the long-term safe operation of the public transportation system. Early warning information is released in a variety of ways, including platform display screens, voice broadcasts, and sending early warning information to bus companies, meeting the needs of different scenarios and audiences. At the same time, the system also allows flexible adjustment of early warning thresholds and rules according to actual conditions, which improves the flexibility and scalability of the system, allowing bus companies to customize the most suitable safety early warning plan according to actual conditions.

[0018] 3. In the process of data processing and model building, refined feature extraction and multi-source data fusion optimization play a vital role. By deeply mining the detailed information in the data, refined feature extraction can reveal the key features hidden in the multi-dimensional data, which are crucial for the identification of abnormal behavior. At the same time, multi-source data fusion optimization further enhances the model's ability to integrate data. It can not only effectively integrate data from different sources, but also eliminate redundancy and conflict between data through advanced algorithms and technologies, and extract more accurate and comprehensive information. This fusion method enables the model to understand the data more comprehensively and provide stronger support for abnormal behavior identification.

[0019] 4. By introducing more advanced algorithms and technologies, the recognition accuracy of the model has been significantly improved. This improvement is not only reflected in the accurate identification of known abnormal behaviors, but also in the keen capture of newly emerging abnormal behaviors. This enables the model to respond to potential hidden dangers more promptly, providing strong protection for the safety and operational efficiency of the public transportation system. At the same time, the close integration with the early warning system enables the early warning system to be activated immediately once the model identifies abnormal behavior, and effectively respond to potential risks by sending early warning information, adjusting operating strategies, etc., to ensure the smooth operation of the public transportation system.

[0020] 5. Applying the DTW dynamic time warping algorithm can accurately align data with different timestamps on the time axis. This step effectively solves the time inconsistency problem of time series data, making the extracted key time series features more accurate and reliable. This is crucial for the subsequent learning and prediction of abnormal behavior recognition models.

[0021] 6. Applying the DTW dynamic time warping algorithm can accurately align data with different timestamps on the time axis. This step effectively solves the time inconsistency problem of time series data, making the extracted key time series features more accurate and reliable. This is crucial for the subsequent learning and prediction of abnormal behavior recognition models. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 The present invention is a flowchart of a method for intelligently analyzing abnormal behavior of platform vehicles based on deep learning according to an embodiment of the present invention. DETAILED DESCRIPTION

[0023] To facilitate the understanding of the present invention, the present application will be described more comprehensively below with reference to the relevant drawings; the drawings show preferred embodiments of the present invention, but the present invention can be implemented in many different forms and is not limited to the embodiments described herein; on the contrary, the purpose of providing these embodiments is to enable a more thorough and comprehensive understanding of the disclosed content of the present invention.

[0024] It should be noted that the terms “vertical”, “horizontal”, “up”, “down”, “left”, “right” and similar expressions used in this document are only for illustrative purposes and do not represent the only implementation method.

[0025] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by technicians in the technical field to which the present invention belongs; the terms used in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention; the term "and / or" used herein includes any and all combinations of one or more related listed items.

[0026] Figure 1 It is a flow chart of a method for intelligently analyzing abnormal behavior of platform vehicles based on deep learning according to an embodiment of the present invention.

[0027] like Figure 1 As shown, a method for intelligent analysis of abnormal behavior of platform vehicles based on deep learning includes the following steps: S101, collect data using vehicle-mounted collection equipment and platform collection equipment to obtain multi-dimensional data.

[0028] Among them, multi-dimensional data includes platform passenger flow information, road condition information, video surveillance information, and vehicle operation information.

[0029] Specifically, GPS locators, acceleration sensors, cameras and other equipment are installed on buses. GPS locators are used to collect vehicle location information, acceleration sensors are used to collect vehicle acceleration information, and cameras are used to collect video surveillance information inside and outside the vehicle; they collect information such as the vehicle's location, speed, acceleration, braking status, number of passengers in the vehicle, and video of the environment outside the vehicle.

[0030] Install passenger flow counters, video surveillance cameras and other equipment at bus stops. Passenger flow counters are used to collect platform passenger flow information, and video surveillance cameras are used to collect platform video surveillance information; collect platform passenger flow, number of passengers getting on and off, platform video surveillance information, and road condition information near the platform (such as traffic flow, road conditions, etc.).

[0031] The platform collection equipment collects data every 5 seconds during peak hours and every 10 seconds during non-peak hours. In actual applications, the collection frequency can be dynamically adjusted according to changes in passenger flow, and the collected data can be transmitted in real time to the data center for processing via the wireless communication network 4G / 5G.

[0032] Among them, platform passenger flow information includes boarding passenger flow, alighting passenger flow and platform waiting passenger flow data; road condition information covers road congestion, road traffic flow, road conditions and construction conditions; video surveillance information includes platform surveillance video and vehicle internal surveillance video; vehicle operation information includes vehicle number, license plate number and vehicle status.

[0033] S102, performing data preprocessing on the collected data.

[0034] Among them, data preprocessing specifically includes: Identify missing values ​​and delete the entire data record containing missing values: For data records with missing values ​​of more than 5 seconds, delete the entire record directly; for other missing values, use linear interpolation to fill in the data; Analyze outliers and delete the entire data record containing outliers: traverse the data set, calculate the mean (μ) and standard deviation (σ) of all valid data points; according to the 3σ principle, set the outlier threshold to the mean plus or minus three times the standard deviation, that is, [μ-3σ,μ+3σ], traverse the data set to check whether each data point exceeds the set outlier threshold range; generally, the mean plus or minus three times the standard deviation can be used as the outlier detection threshold. For data records that exceed the threshold, perform manual review or make corrections based on business logic; if correction is not possible, delete the entire record; Detect duplicate data and delete redundant data: Use hash algorithms to detect duplicate records in the data, delete redundant records, and only retain unique records; Convert timestamp data to a standard format: Convert timestamp data collected by different devices to the UTC time format to ensure the consistency of timestamps.

[0035] Remove non-numeric characters from numerical data: Remove non-numeric characters from numerical data, such as spaces, commas, etc. For abnormal values ​​(such as too large, too small, negative, etc.), perform manual review and correction.

[0036] Convert text data into a unified format and remove spaces: Convert text data into a unified UTF-8 encoding format and remove spaces, special characters, and extra punctuation marks; directly delete garbled or invalid characters in text data; Convert location data into a unified coordinate system: Convert the collected longitude and latitude location data into a unified WGS-84 coordinate system to facilitate subsequent spatial analysis.

[0037] For the above-mentioned existing technologies such as linear interpolation, 3σ principle, hash algorithm, etc., please refer to the step flow of the existing technology, which will not be described in detail in this article.

[0038] S103, performing data fusion on the pre-processed data to form multi-dimensional safety warning indicators.

[0039] Specifically, data from different devices and in different formats are aligned based on timestamps to ensure data synchronization; Use timestamp synchronization technology and geographic coordinate conversion methods to ensure accurate alignment of data sources in time and space; Extract key features from the aligned data, such as vehicle speed, acceleration, braking status, number of passengers at the platform, number of passengers getting on and off the bus, etc. Set the initial state vector X and state covariance matrix P, which are usually set based on prior knowledge or experience; Use the state transfer equation to predict the state vector X' and state covariance matrix P' at the next moment; The state transition equations are usually modeled based on the physical model or dynamic characteristics of the system; When new observation data arrive, use the observation equation to transform this data into the same dimension as the state vector; Calculate the Kalman gain K, which reflects the degree of correction of the state estimate by the observed data; Use the Kalman gain to update the state vector X and the state covariance matrix P to obtain the fused state estimate; Repeat the prediction and update phases to continuously incorporate new observation data to obtain a more accurate state estimate; In the public transportation safety warning system, the vehicle's position, speed and other information can be used as a state vector, and the Kalman filter algorithm can be used to fuse data from different sensors to improve the accuracy and reliability of positioning; The construction process of constructing multi-dimensional safety warning indicators takes the construction process of vehicle speeding warning indicators, passenger congestion warning indicators, and road congestion warning indicators as examples: According to road traffic regulations or actual road conditions, a speed threshold for the vehicle is set; the vehicle speed information is extracted from the fused data, and the current speed of the vehicle is compared with the set threshold; if the vehicle speed exceeds the threshold, a speeding warning is triggered.

[0040] Set the threshold of passenger crowding according to factors such as platform capacity and passenger comfort; extract platform passenger flow information from the fused data; compare the current platform passenger flow with the set threshold, and trigger a passenger crowding warning if the passenger flow exceeds the threshold; The threshold of road congestion is set according to factors such as traffic flow and road conditions. This can be a comprehensive indicator that takes into account the influence of multiple factors; extract information such as traffic flow and road conditions from the fused data; comprehensively analyze traffic flow and road conditions, and if the set congestion threshold is reached or exceeded, a road congestion warning is triggered.

[0041] Through the above process, a multi-dimensional and comprehensive safety early warning indicator can be constructed.

[0042] S104, based on the fused data, a public transportation safety warning model is constructed, and the model is used to issue abnormal behavior warnings.

[0043] Among them, the public transportation safety warning model is built using a deep learning algorithm, which can automatically learn and identify abnormal behavior patterns of public transportation vehicles.

[0044] Specifically, the fused data is divided into a training set (70%), a validation set (15%), and a test set (15%). Multiple convolution kernels are set to extract features from the data. The convolution kernel size is set to 3x3 (which can be adjusted according to feature extraction requirements), the number of convolution kernels is 32 (which can be adjusted according to model complexity), and the step size and other parameters are determined based on experiments. The ReLU function is used as the activation function to increase the nonlinear expression ability of the model, and the maximum pooling is used to reduce the dimension of the data and reduce the amount of calculation; Combine the features extracted by the convolutional layer to form the final prediction result; The number of neurons in the output layer is set according to the specific task, such as one neuron for a binary classification problem and multiple neurons for a multi-classification problem.

[0045] Randomly initialize the weights and biases of the CNN model, set the learning rate to 0.001 (can be adjusted according to actual conditions), batch size to 64 (can be adjusted according to the size of the dataset and computing power), and the number of iterations to 100 (can be adjusted according to the convergence of the model) and other training parameters; The training data is input into the CNN model, and the final prediction result is obtained through layer-by-layer calculations such as convolutional layer, activation layer, and pooling layer. Use the cross entropy loss function to calculate the error between the predicted result and the true label; According to the gradient of the loss function, the weights and biases of the CNN model are updated using the back-propagation algorithm; Repeat the process of forward propagation, loss calculation, and back propagation until the preset number of iterations or the convergence condition of the loss function is reached; Deploy the trained model to the server and provide prediction services through the API interface.

[0046] S105, releasing the warning information through various means.

[0047] Among them, the release methods include platform display screens, voice broadcasts and sending early warning information to bus companies.

[0048] For example, using a GPS locator on a bus collects the vehicle's location information once a second, including longitude and latitude, with an accuracy of within 5 meters; The acceleration sensor collects the acceleration information of the vehicle once a second to determine the driving status of the vehicle (such as sudden acceleration, sudden braking, etc.); Two cameras are installed in the car to monitor the interior and exterior environment respectively, with a frame rate of 30 frames per second and a resolution of 1080P; Through infrared sensor technology, the number of passengers in the car is updated every second; At the bus station, a passenger flow counter is installed at the entrance of the station. Using video recognition technology, data is collected every 5 seconds during peak hours and every 10 seconds during non-peak hours to count the number of passengers on the platform and the number of passengers getting on and off the bus. Two cameras are installed to cover the platform area and nearby road conditions, with a frame rate of 30 frames per second and a resolution of 720P; Traffic flow and road condition information (such as congestion, accidents, etc.) near the platform are collected every 10 seconds through road condition information sensors and image processing technology; The platform data collection equipment dynamically adjusts the collection frequency according to the changes in passenger flow. Data is collected every 5 seconds during peak hours and every 10 seconds during non-peak hours. All collected data is transmitted to the data center in real time via the 4G / 5G wireless communication network, with a transmission delay of no more than 2 seconds. According to road traffic regulations or actual road conditions, set the vehicle speed threshold (such as 60km / h). Extract the vehicle speed information from the fused data and compare the vehicle's current speed with the set threshold. If the vehicle speed exceeds the threshold, a speeding warning is triggered; According to factors such as platform capacity and passenger comfort, set the threshold of passenger congestion (80% of platform capacity). Extract platform passenger flow information from the fused data and compare the current platform passenger flow with the set threshold. If the passenger flow exceeds the threshold, a passenger congestion warning is triggered; According to factors such as traffic flow and road conditions, set the threshold of road congestion (for example, if the traffic flow is lower than the set threshold and the road condition is "congested"). Extract traffic flow and road condition information from the fused data, and after comprehensive analysis, if it reaches or exceeds the set congestion threshold, trigger a road congestion warning; Install LED screens at bus stops to display real-time warning information, such as "The vehicle is speeding, please pay attention to safety!", "The platform is crowded, please board the bus in an orderly manner!", etc. The display screen update frequency should be consistent with the change frequency of the warning information; Warning information is broadcast through the platform broadcasting system to remind passengers to pay attention to safety. The voice broadcast content is clear and accurate, and is updated synchronously with the information on the display screen; Send early warning information to bus companies via SMS or email, including abnormal situations such as vehicle speeding, excessive passenger crowding, road congestion, etc. Bus companies can adjust their operating strategies in a timely manner based on the early warning information to improve the safety and reliability of bus services.

[0049] The technical solutions in the above embodiments of the present application have at least the following technical effects or advantages: 1. By collecting multi-dimensional data including platform passenger flow, road conditions, video surveillance and vehicle operation information, this technical solution provides a comprehensive insight into the operating status of the public transportation system. Data fusion technology further integrates these scattered information sources, eliminates data redundancy and contradictions, and thus generates more accurate and reliable safety warning indicators, providing a solid foundation for public transportation safety management. Automated data collection, preprocessing and fusion processes, combined with efficient early warning model calculations, ensure that the system can respond to any abnormal situation in the public transportation system in real time. This real-time nature not only improves the timeliness of early warnings, but also enables relevant departments and passengers to quickly take necessary countermeasures, effectively reducing the probability of safety accidents.

[0050] 2. The early warning model built with deep learning algorithms can automatically learn and identify abnormal behavior patterns of public transportation vehicles. This intelligent feature enables the model to continuously adapt to the dynamic changes of the public transportation system, improve the accuracy and precision of early warnings through continuous learning and optimization, and provide strong guarantees for the long-term safe operation of the public transportation system. Early warning information is released in a variety of ways, including platform display screens, voice broadcasts, and sending early warning information to bus companies, meeting the needs of different scenarios and audiences. At the same time, the system also allows flexible adjustment of early warning thresholds and rules according to actual conditions, which improves the flexibility and scalability of the system, allowing bus companies to customize the most suitable safety early warning plan according to actual conditions.

[0051] Example 2: By collecting multi-dimensional data such as platform passenger flow information, road condition information, video surveillance information, and vehicle operation information, a comprehensive data foundation is provided to more accurately reflect the actual operation status of the public transportation system. In order to further improve the safety and operational efficiency of the public transportation system and more effectively deal with potential hidden dangers, more advanced feature extraction and fusion algorithms are introduced through the refinement of the data fusion process to achieve higher-precision abnormal behavior detection.

[0052] Now, further improvements are made on the basis of step S103 in the first embodiment, specifically: S201, use graph neural network to perform preliminary fusion of preprocessed data.

[0053] Specifically, graph neural networks represent the relationship between data by constructing a graph structure between data and extract features using convolution operations; Extract features from pre-processed data (including platform passenger flow information, road condition information, video surveillance information, and vehicle operation information), which can be achieved through existing deep learning models; In graph neural networks, an attention mechanism is introduced. In each layer of the graph neural network or a specific aggregation step, an attention score for each feature is calculated. This is usually done by taking a dot product of the feature vector with a learnable attention weight vector; Next, the attention scores are converted into weights using the softmax function, and these weights are applied to the feature aggregation process; Then, the graph neural network model is trained using labeled training data (including samples of normal and abnormal behavior). During the training process, the model learns how to identify abnormal behavior based on input features and attention mechanisms; After training, the graph neural network outputs preliminary fused feature representations that not only integrate information from different dimensions and entities, but also emphasize features related to abnormal behavior identification.

[0054] S202, performing multi-source data fusion optimization on the fusion result of the graph neural network.

[0055] Specifically, in addition to the preliminary fusion results of the graph neural network, data from other sources will also be prepared, such as directly collected road condition information, original frames of video surveillance, or feature extraction results; Necessary preprocessing will be performed on these data, such as time synchronization, denoising, and normalization, to ensure that they are consistent with the output of the graph neural network in terms of dimension and scale; Next, the Kalman filter algorithm is applied to perform preliminary fusion of multi-source data. On the basis of Kalman filter fusion, GNN (graph neural network) is further introduced to perform deep fusion of multi-source data, and the forward propagation formula of GNN H(l+1)=σ(AH(l)W(l)) is used to update node features. Among them, A is the adjacency matrix, which represents the relationship between nodes; H(l) is the node feature of the lth layer; W(l) is the learnable weight matrix; σ is the activation function, which is used to introduce nonlinear characteristics; For time series data such as video surveillance information, the original convolutional neural network (CNN) model is upgraded to a hybrid model combining long short-term memory network (LSTM) and CNN. LSTM can handle long-term dependencies in time series data, while CNN is good at capturing spatial features. Combining the two can better capture the temporal and spatial features in the video, thereby improving the accuracy of abnormal behavior recognition; Subsequently, the labeled training data is used to train the entire fusion model, including GNN and hybrid model (LSTM+CNN). During the training process, the model parameters are optimized through the back propagation algorithm to minimize the loss function and improve the accuracy of abnormal behavior recognition; After training is completed, new multi-source data is input into the fusion model for abnormal behavior detection. Based on the output of the model, potential abnormal behaviors can be identified and corresponding response mechanisms can be triggered, such as sending early warning information to the bus company, displaying warnings on the platform display, etc.

[0056] For the above-mentioned fusion and combination of models, please refer to the existing technology, which will not be elaborated in this article.

[0057] The technical solutions in the above embodiments of the present application have at least the following technical effects or advantages: In the process of data processing and model building, refined feature extraction and multi-source data fusion optimization play a vital role. By deeply mining the detailed information in the data, refined feature extraction can reveal the key features hidden in the multi-dimensional data, which are crucial for the identification of abnormal behavior. At the same time, multi-source data fusion optimization further enhances the model's ability to integrate data. It can not only effectively integrate data from different sources, but also eliminate redundancy and conflict between data through advanced algorithms and technologies, and extract more accurate and comprehensive information. This fusion method enables the model to understand the data more comprehensively and provide stronger support for abnormal behavior identification.

[0058] By introducing more advanced algorithms and technologies, the recognition accuracy of the model has been significantly improved. This improvement is not only reflected in the accurate identification of known abnormal behaviors, but also in the keen capture of newly emerging abnormal behaviors. This enables the model to respond to potential hidden dangers more promptly, providing strong protection for the safety and operational efficiency of the public transportation system. At the same time, the close integration with the early warning system enables the early warning system to be activated immediately once the model identifies abnormal behavior, and effectively respond to potential risks by sending early warning information, adjusting operating strategies, etc., to ensure the smooth operation of the public transportation system.

[0059] Example 3: The refined feature extraction and multi-source data fusion optimization of the above Example 2 enable the model to more accurately capture and fuse key information in multi-dimensional data, providing stronger support for abnormal behavior identification. At the same time, the upgrade of the abnormal behavior identification model further improves the accuracy of identification, allowing the early warning system to respond to potential hidden dangers more promptly. In order to further improve the accuracy and efficiency of data fusion, an adaptive graph convolutional network and a dynamic time warping algorithm are introduced to further improve the indicators of the final effect.

[0060] Now, further improvements are made on the basis of step S202 in the second embodiment, specifically: S301, introduce the AGCN adaptive graph convolutional network and adjust the parameters of the convolution kernel according to the actual distribution of the data.

[0061] Specifically, first, the original multi-source data is preprocessed, including data cleaning, denoising, normalization, etc., to ensure the quality and consistency of the data; Convert the preprocessed data into a graph structure. Nodes represent data points (such as vehicles, passengers, etc.), and edges represent the relationship between data points (such as vehicle driving trajectories, passenger flow, etc.). This step requires defining the construction rules of nodes and edges, as well as how to assign attributes to each node and edge (such as timestamps, location information, etc.); Design multiple layers of graph convolutional layers, each of which contains learnable convolution kernel parameters. These convolution kernels will be used to perform convolution operations on the graph structure to extract local features of nodes; Assign an attention weight to each node, which is calculated based on the node's features and the features of its neighboring nodes. These weights can be used to adjust the parameters of the convolution kernel when applied to different nodes; Design additional neural network fully connected layers to dynamically generate convolution kernel parameters based on the features of the input graph. These networks can receive global or local features of the graph as input and output adjusted convolution kernel parameters; After the graph convolution layer, a graph pooling layer can be introduced to reduce the number of nodes while retaining important feature information. This helps reduce computational complexity and extract more representative features; Through the adaptive graph convolution layer, high-level features of the data can be extracted, and effective feature fusion strategies can be adopted to integrate these features at different levels.

[0062] S302, applying the DTW dynamic time warping algorithm to accurately align data with different timestamps on the time axis.

[0063] Specifically, firstly, the time series data such as vehicle operation status data are preprocessed, which includes denoising, smoothing and other steps. Then, the DTW algorithm is applied to accurately align these time series data on the time axis to deal with the problem of data inconsistency in time. In the aligned time series data, key time series features can be extracted, which will be used for subsequent abnormal behavior identification.

[0064] Then, an abnormal behavior recognition model is built based on deep learning technology, which combines convolutional neural network (CNN) and long short-term memory network (LSTM). The features extracted by AGCN module and DTW module are used as input to this deep learning model for abnormal behavior recognition.

[0065] During the training phase, the deep learning model is trained and optimized using labeled training data, aiming to improve the accuracy and generalization ability of abnormal behavior recognition.

[0066] After training is completed, new data is input into the model for real-time abnormal behavior recognition. Once the model recognizes abnormal behavior, the system will immediately trigger the early warning mechanism and send early warning information to relevant personnel.

[0067] Finally, based on the type and severity of abnormal behavior, corresponding emergency response strategies will be formulated and implemented, such as adjusting vehicle scheduling and strengthening platform monitoring.

[0068] The technical solutions in the above embodiments of the present application have at least the following technical effects or advantages: By introducing AGCN, the scheme can dynamically adjust the parameters of the convolution kernel according to the actual distribution of the data, so as to more flexibly capture the spatiotemporal features and complex relationships in the data. This adaptability enables the model to more effectively handle different scenarios and changing data patterns, improving the accuracy and robustness of feature extraction.

[0069] By applying the DTW dynamic time warping algorithm, data with different timestamps can be accurately aligned on the time axis. This step effectively solves the time inconsistency problem of time series data, making the extracted key time series features more accurate and reliable. This is crucial for the subsequent learning and prediction of abnormal behavior recognition models.

[0070] By integrating deep learning technologies such as convolutional neural networks (CNN) and long short-term memory networks (LSTM), a powerful abnormal behavior recognition model is built. By taking the features extracted by the AGCN module and the DTW module as input, the model can learn deeper feature representations, thereby improving the accuracy and generalization ability of abnormal behavior recognition.

[0071] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for intelligent analysis of abnormal behavior of platform vehicles based on deep learning, characterized in that: include: S101, collect data using vehicle-mounted collection equipment and platform collection equipment to obtain multi-dimensional data; S102, performing data preprocessing on the collected data; S103, performing data fusion on the pre-processed data to form a multi-dimensional safety early warning indicator; S104, based on the fused data, construct a public transportation safety warning model, and use the model to issue abnormal behavior warnings; S105, releasing the warning information through various means.

2. The method for intelligent analysis of abnormal behavior of platform vehicles based on deep learning as claimed in claim 1, characterized in that: The step S101, obtaining multi-dimensional data, includes: Platform passenger flow information, road condition information, video surveillance information and vehicle operation information.

3. The method for intelligent analysis of abnormal behavior of platform vehicles based on deep learning as claimed in claim 1, characterized in that: The step S102, performing data preprocessing on the collected data, includes: Identify missing values ​​and delete entire data records containing missing values; Analyze outliers and delete the entire data record containing the outlier; Detect duplicate data and delete redundant data; Convert timestamp data to UTC time format; Remove non-numeric characters from numeric data; Convert text data to the unified UTF-8 encoding format and remove spaces; Convert the position data to the unified WGS-84 coordinate system.

4. The method for intelligent analysis of abnormal behavior of platform vehicles based on deep learning as claimed in claim 1, characterized in that: The step S104, constructing a public transportation safety early warning model, includes: Built with deep learning algorithms, it can automatically learn and identify abnormal behavior patterns of public transportation vehicles.

5. The method for intelligent analysis of abnormal behavior of platform vehicles based on deep learning as claimed in claim 1, characterized in that: The S103 includes: S201, using graph neural network to perform preliminary fusion of preprocessed data; S202, performing multi-source data fusion optimization on the fusion result of the graph neural network.

6. The method for intelligent analysis of abnormal behavior of platform vehicles based on deep learning as claimed in claim 5, characterized in that: The S102 includes: Perform feature extraction on the preprocessed data; Introducing the attention mechanism in graph neural networks; At each layer of the graph neural network or at a specific aggregation step, the attention score for each feature is calculated; Use the softmax function to convert the attention scores into weights and apply these weights to the feature aggregation process; Use labeled sample training data including normal and abnormal behaviors to train the graph neural network model; During training, the model learns how to identify abnormal behaviors based on input features and attention mechanisms.

7. The method for intelligent analysis of abnormal behavior of platform vehicles based on deep learning as claimed in claim 5, characterized in that: The S202 includes: In addition to the preliminary fusion results of the graph neural network, data from other sources, such as directly collected road condition information, original frames of video surveillance, or feature extraction results, are called new multi-source data; Time synchronization, denoising, and normalization of new multi-source data; Apply Kalman filter algorithm to initially fuse multi-source data; On the basis of Kalman filter fusion, GNN graph neural network is further introduced to perform deep fusion of multi-source data, and the forward propagation formula of GNN H(l+1)=σ(AH(l)W(l)) is used to update node features; where A is the adjacency matrix, which represents the relationship between nodes; H(l) is the node feature of the lth layer; W(l) is a learnable weight matrix; σ is an activation function, which is used to introduce nonlinear characteristics; For time series data, the original CNN convolutional neural network model is upgraded to a hybrid model combining LSTM long short-term memory network and CNN; Use labeled training data to train the entire fusion model, including GNN and hybrid model; After training is completed, new multi-source data is input into the fusion model for abnormal behavior detection.

8. The method for intelligent analysis of abnormal behavior of platform vehicles based on deep learning as claimed in claim 5, characterized in that: The S202 further includes: S301, introduce the AGCN adaptive graph convolutional network and adjust the parameters of the convolution kernel according to the actual distribution of the data; S302, applying the DTW dynamic time warping algorithm to accurately align data with different timestamps on the time axis.

9. The method for intelligent analysis of abnormal behavior of platform vehicles based on deep learning as claimed in claim 8, characterized in that: The S301 includes: Preprocess the original multi-source data; The preprocessed data is converted into a graph structure representation; the nodes represent data points, vehicles and passengers, and the edges represent the relationship between vehicle driving trajectories and passenger flow data points; Design multiple layers of graph convolutional layers, each of which contains learnable convolution kernel parameters; Assign an attention weight to each node, which is calculated based on the node's features and the features of its neighboring nodes; Design additional neural network fully connected layers to dynamically generate convolution kernel parameters based on the characteristics of the input graph; After the graph convolution layer, a graph pooling layer is introduced to reduce the number of nodes while retaining important feature information.

10. The method for intelligently analyzing abnormal behaviors of platform vehicles based on deep learning as claimed in claim 8, characterized in that: The S302 includes: Preprocess the time series data of platform passenger flow information, road condition information, video surveillance information and vehicle operation information data by denoising and smoothing; Apply the DTW algorithm to accurately align these time series data on the time axis; Extract key time series features from the aligned time series data; Build an abnormal behavior recognition model based on deep learning technology, which combines CNN convolutional neural network and LSTM long short-term memory network; The features extracted by the AGCN module and the DTW module are used as input to this deep learning model for abnormal behavior recognition.

Images