Analysis system based on driver behavior

Abstract

The invention discloses an analysis system based on vehicle captain behaviors. According to the invention, the behavior recognition module adopts a deep learning model, a convolutional neural network (CNN) and a recurrent neural network (RNN), can automatically learn features in data, and does not need to manually design the features. This enables the model to more accurately identify the driving habit, service level, mental appearance, standard driving and other behaviors of the driver. The deep learning model can carry out end-to-end learning, directly learns a behavior recognition model from original data, and does not need to carry out feature engineering. The model training process is simplified, and the model performance is improved. And the recognition module connects the output features of the CNN and the RNN to form a final feature vector for behavior recognition. The multi-modal feature fusion mode can describe the behaviors of the driver more comprehensively, so that the accuracy of behavior recognition is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of automobile safety behavior analysis, and in particular is an analysis system based on vehicle leader behavior. Background Art

[0002] With the rapid development of the transportation industry, the size of the fleet continues to expand, and fleet management faces more and more challenges. As an important part of the fleet, the driver's driving behavior directly affects driving safety and passenger experience. The traditional fleet management model mainly relies on manual supervision and experience judgment, which has problems such as low efficiency, strong subjectivity, and difficulty in quantification. In recent years, with the rapid development of artificial intelligence technology, driving behavior recognition algorithms based on deep learning have gradually been applied to the field of fleet management. These algorithms can automatically identify the driver's driving behavior, such as whether to give way to pedestrians, whether to wear seat belts, etc., so as to help fleet managers better understand the driver's working status and conduct targeted training and management.

[0003] However, the existing analysis system based on the driver's behavior mainly relies on vehicle-mounted monitoring equipment, which makes it difficult to comprehensively collect the driver's driving behavior data. At the same time, it mainly adopts simple threshold method or classification method, which makes it difficult to comprehensively evaluate the driver's behavior performance. Summary of the invention

[0004] The purpose of the present invention is to provide an analysis system based on the behavior of the vehicle captain in order to solve the above-mentioned problems.

[0005] The technical solution adopted by the present invention is as follows: an analysis system based on the behavior of the vehicle leader, the system comprising: a data acquisition module, a behavior recognition module, an evaluation module, a reporting module and a management module;

[0006] The behavior recognition module is internally provided with a data preprocessing module, a feature extraction module, a behavior recognition module and a post-processing module;

[0007] The data acquisition module is the starting point of the entire driver behavior analysis system. It is responsible for collecting the driver's driving behavior data from the vehicle monitoring equipment, GPS positioning system, and vehicle terminal equipment, including driving behavior data, voice data, and video data; these data are cleaned, enhanced, and annotated by the data preprocessing module and then input into the behavior recognition module;

[0008] The behavior recognition module is the core of the system. It uses a combination model of convolutional neural network (CNN) and recurrent neural network (RNN) to extract image features and time series features respectively, and perform behavior recognition. CNN is used to extract image features in video frames, including the driver's actions and expressions; RNN is used to extract time series features in video frame sequences, including the duration and frequency of the driver's behavior. The behavior recognition module connects the output features of CNN and RNN, and uses a fully connected layer to map the features to behavior categories, thereby identifying the driver's behavior.

[0009] The evaluation module evaluates the various behaviors of the driver based on the results of the behavior recognition module and calculates a comprehensive score; the evaluation module performs a weighted average of the scores of the various behaviors of the driver to obtain a comprehensive score of the driver, and classifies the driver into different levels according to the comprehensive score, including enhanced assistance, qualified, and excellent;

[0010] The report module generates a driver behavior analysis report based on the evaluation results, and the report content includes the driver's basic information, various behavior analysis results, comprehensive scores, levels and improvement suggestions; the report module presents the driver's behavior analysis results to the fleet managers in a visual manner to help them better understand the driver's work status;

[0011] The management module provides targeted training and management for the driver according to the driver behavior analysis report; the management module formulates personalized training plans and management programs according to the driver's level and improvement suggestions, including driving skills training for the driver with poor driving habits, service awareness training for the driver with poor service level, psychological counseling for the driver with poor mental outlook, and traffic regulations training for the driver with poor driving standards.

[0012] In a preferred embodiment, the data acquisition module collects the driver's driving behavior data from the vehicle-mounted monitoring equipment, GPS positioning system, and vehicle-mounted terminal equipment, including driving behavior data, voice data, and video data; these data are used to analyze the driver's driving habits, service level, mental outlook, standardized driving, and other behaviors;

[0013] Driving behavior data includes: vehicle speed, acceleration, steering angle, and braking force, which are used to analyze the driver's driving habits, including whether he changes lanes frequently and runs yellow lights;

[0014] Voice data includes: the conversation between the driver and the passengers, which is used to analyze the driver's service level, including whether he uses civilized language and whether he actively serves passengers;

[0015] Video data alarm: The driver's actions and expressions during driving are used to analyze the driver's mental state, standardized driving and other behaviors, including whether he wears a seat belt and whether he gives way to pedestrians.

[0016] In a preferred embodiment, the data preprocessing module uses a minimum-maximum standardization method to perform data standardization processing, and the specific steps are as follows:

[0017] S1. Calculate the maximum and minimum values: For each feature, find the maximum and minimum values ​​in the Internet security information dataset;

[0018] S2. Apply transformation formula: For each feature value x in the Internet security information dataset, use the following formula for transformation:

[0019]

[0020] Among them, x norm is the transformed value;

[0021] S3. Transform the Internet security information dataset: Apply the above transformation to each feature of the entire dataset; after minimum-maximum normalization, all features will be scaled to the range of 0 to 1.

[0022] In a preferred embodiment, the feature extraction module extracts features that are helpful for model learning from the preprocessed data; the specific content and operation method are as follows: In this module, first, the features with high correlation with security events are screened out through feature selection technology, and then the principal component analysis feature conversion method is used to convert the data to make it more suitable for machine learning model processing; for network traffic data, the feature extraction module will calculate the traffic size, protocol type, and entropy value of the source / destination IP address of the data packet, and use these feature vectors as input for subsequent AI models.

[0023] In a preferred embodiment, the identification content of the behavior identification module includes:

[0024] Driving habit analysis:

[0025] Identify the driver’s driving habits based on driving behavior data, including:

[0026] Do you give way to pedestrians?

[0027] Doing things unrelated to driving?

[0028] Are you wearing a seat belt?

[0029] Do you drive with one hand?

[0030] Whether to touch the phone

[0031] Record the number of times, common locations, and common times of each behavior;

[0032] Service Level Analysis:

[0033] Identify the captain’s service level based on voice and video data, including:

[0034] Whether to use civilized language

[0035] Whether to proactively serve passengers

[0036] Indifferent service

[0037] Whether there is quarrel / swearing

[0038] Clean the carriage?

[0039] Whether to dress properly and wear a hat

[0040] Record the number and time period of each behavior;

[0041] Mental outlook analysis:

[0042] Identify the driver's mental state based on video data, including:

[0043] Whether to actively send safety alarm

[0044] Whether you are driving while fatigued

[0045] Is the sitting posture improper?

[0046] Is it difficult to concentrate?

[0047] Drink water / eat food?

[0048] Record the number, time period, and frequent locations of each behavior; standardize driving analysis:

[0049] Identify the driver's standard driving behavior based on driving behavior data, including: whether to run a yellow light

[0050] Do you give way to pedestrians?

[0051] Is it an aggressive signal?

[0052] Take the fast lane?

[0053] Whether passengers are picked up or dropped off outside the station

[0054] Whether to press the line

[0055] Whether the station is not in compliance with regulations

[0056] Do you change lanes frequently?

[0057] Whether the zebra crossing does not slow down

[0058] Overtaking in the next lane

[0059] Did you fail to drive in the guide lane?

[0060] Whether the road is flooded and the speed is not reduced

[0061] Record the number of times each behavior occurs and the locations where it is often performed;

[0062] Other behavioral analyses:

[0063] Other behaviors of the driver can be identified based on video data, including:

[0064] Do you listen to the radio?

[0065] Whether the vehicle honks

[0066] Whether to throw objects out of the car window

[0067] Whether the behavior is indecent

[0068] Whether you leave the vehicle without explaining the reason to the passengers

[0069] Did you change lanes on the left without giving a thumbs up?

[0070] Did you turn right without stopping to look?

[0071] Is the passenger not sitting properly and holding the car steady?

[0072] Record the number, duration, reason, and location of each behavior.

[0073] In a preferred embodiment, the behavior recognition module uses 70% of the vehicle captain's behavior data as a training set, 15% of the vehicle captain's behavior data as a validation set, and 15% of the vehicle captain's behavior data as a test set;

[0074] The model training module uses the training set to train the model, optimizes the performance by adjusting the model parameters, and uses the batch gradient descent algorithm to optimize the parameters of the logistic regression model. The iterative update formula is:

[0075] Where θj is the model parameter, α is the learning rate, and J(θ) is the cross entropy loss function, which is calculated as:

[0076]

[0077] Where m is the number of training samples, y(i) is the true label of the i-th sample, and h θ (x (i)) is the prediction result of the model;

[0078] The behavior recognition module then first determines the basic architecture of the model by comparing the performance of different algorithms on the validation set, and then uses hyperparameter tuning technology to find the optimal parameter combination. The model selection and optimization submodule also uses cross-validation methods to evaluate the stability of the model on different data subsets, thereby ensuring that the selected model has good generalization capabilities.

[0079] In a preferred embodiment, the behavior recognition module uses a normal driver behavior data set to train a model and determine the K nearest neighbors of each driver behavior data point; then, for a new driver behavior data point, the average distance between it and the K nearest neighbors is calculated; if the average distance exceeds a preset threshold, the driver behavior data point is marked as abnormal;

[0080] The calculation formula for anomaly detection is:

[0081]

[0082] Among them: the anomaly score indicates the degree of abnormality of the driver's behavior data point. The higher the score, the greater the possibility of abnormality;

[0083] k represents the number of selected nearest neighbors, that is, the K value in K-NN;

[0084] d(data,neighbor i ): represents the distance between the driver behavior data point and its i-th nearest neighbor; using Euclidean distance as the distance metric, the calculation formula is:

[0085]

[0086] in:

[0087] n represents the number of features of the captain's behavior data points;

[0088] x j,data Represents the value of the current driver behavior data point on the jth feature;

[0089] xj,neighbori represents the value of the i-th nearest neighbor on the j-th feature;

[0090] In this way, the behavior recognition module can automatically learn the distribution characteristics of normal driver behavior data, and identify behaviors that do not conform to normal patterns without prior labeling of abnormal driver behavior data, thereby effectively responding to changes in abnormal driver behavior.

[0091] In a preferred embodiment, the evaluation module evaluates each behavior of the driver based on the result of the behavior recognition module and calculates a comprehensive score; the evaluation module performs a weighted average of the scores of each behavior of the driver to obtain a comprehensive score of the driver, and classifies the driver into different levels according to the comprehensive score, including enhanced assistance, qualified, and excellent;

[0092] Behavior evaluation: The evaluation module evaluates the driver's various behaviors, including whether he gives way to pedestrians, whether he wears a seat belt, and whether he uses civilized language. The evaluation module uses different evaluation methods, including:

[0093] Threshold method: set a threshold for each behavior, and deduct points if the threshold is exceeded;

[0094] Classification: Classify behaviors into different categories, including yielding to pedestrians, not yielding to pedestrians, wearing seat belts, and not wearing seat belts, and assign different scores to each category;

[0095] Comprehensive score calculation: The evaluation module takes a weighted average of the scores of the driver's various behaviors to obtain the driver's comprehensive score; the weight is adjusted according to the actual situation of the fleet, including setting a higher weight for pedestrian-friendly behavior;

[0096] Level division: The evaluation module divides the captains into different levels according to their comprehensive scores, including enhanced assistance, qualified, and excellent. The level division standards are adjusted according to the actual situation of the fleet, including classifying captains with comprehensive scores below 60 points as enhanced assistance.

[0097] In a preferred embodiment, the reporting module generates a driver behavior analysis report based on the evaluation results, and the report content includes the driver's basic information, various behavior analysis results, comprehensive scores, levels and improvement suggestions; the reporting module presents the driver's behavior analysis results to the fleet manager in a visual manner to help them better understand the driver's work status;

[0098] Basic information: including the driver's name, gender, branch, fleet, and vehicle number;

[0099] Behavior analysis results: including the number of times the driver committed each behavior, the score, and the reason for the deduction;

[0100] Comprehensive score: including the comprehensive score and level of the captain;

[0101] Improvement suggestions: including suggestions for improvement on the driver's behavior, including suggesting that the driver should strengthen his awareness of giving way to pedestrians and regulate his driving behavior;

[0102] Visual presentation: The report module uses charts and tables to present the results of the driver's behavior analysis in a visual way, including using bar charts to show the number of each behavior of the driver, and using line charts to show the trend of changes in the driver's comprehensive score.

[0103] In a preferred embodiment, the management module provides targeted training and management for the driver according to the driver behavior analysis report; the management module formulates personalized training plans and management programs according to the driver's level and improvement suggestions, including driving skills training for drivers with poor driving habits, service awareness training for drivers with poor service levels, psychological counseling for drivers with poor mental outlook, and traffic law training for drivers with poor standard driving;

[0104] Training plan: The management module formulates personalized training plans based on the driver's behavioral problems, including organizing driving skills training courses for drivers with poor driving habits, including safe driving training and emergency response training;

[0105] Management plan: The management module formulates different management plans according to the level of the captain, including strengthening supervision and management of captains at the help level, including regular driving behavior inspections and one-on-one conversations with the captain;

[0106] Incentive mechanism: The management module establishes an incentive mechanism to encourage drivers to develop good driving habits and service awareness, including rewards for excellent drivers, such as bonuses and promotions.

[0107] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0108] 1. In the present invention, the behavior recognition module adopts a deep learning model, a convolutional neural network (CNN) and a recurrent neural network (RNN), which can automatically learn features in the data without manually designing features. This enables the model to more accurately identify the driver's driving habits, service level, mental outlook, standardized driving and other behaviors. The deep learning model can perform end-to-end learning and learn the behavior recognition model directly from the raw data without feature engineering. This simplifies the model training process and improves model performance. The output features of CNN and RNN are connected for the recognition module to form a final feature vector for behavior recognition. This multimodal feature fusion method can more comprehensively describe the driver's behavior, thereby improving the accuracy of behavior recognition.

[0109] 2. In the present invention, the output results of the behavior recognition module provide a data basis for the evaluation module and the reporting module. Fleet managers can formulate more scientific management strategies based on these data, such as formulating different training plans and management plans for drivers of different levels, thereby improving management efficiency. The behavior recognition module can help fleet managers understand the behavioral problems of drivers, and thus formulate personalized training plans, such as driving skills training for drivers with poor driving habits, and service awareness training for drivers with poor service levels. This can improve training effectiveness and reduce training costs. The behavior recognition module can identify potential risk factors, such as fatigue driving, irregular driving, etc., and fleet managers can take corresponding measures to control them, such as strengthening the monitoring and management of fatigue driving behavior, thereby reducing risks and improving fleet efficiency. Therefore, the specific algorithm of the internal module of the behavior recognition module can accurately identify the behavior of the driver and provide valuable insights for fleet managers, thereby improving the safety and service quality of the fleet and reducing management costs and risks. BRIEF DESCRIPTION OF THE DRAWINGS

[0110] Figure 1 is the overall system block diagram of the present invention;

[0111] Figure 2 This is a system block diagram of the behavior recognition module in the present invention. DETAILED DESCRIPTION

[0112] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0113] Example:

[0114] Reference Figure 1-2 ,An analysis system based on the captain's behavior, the system includes: a data acquisition module, a behavior recognition module, an evaluation module, a reporting module and a management module;

[0115] The behavior recognition module is internally provided with a data preprocessing module, a feature extraction module, a behavior recognition module and a post-processing module;

[0116] The data collection module is the starting point of the entire driver behavior analysis system. It is responsible for collecting the driver's driving behavior data from the vehicle monitoring equipment, GPS positioning system, vehicle terminal and other equipment, including driving behavior data, voice data and video data. After being cleaned, enhanced and labeled by the data preprocessing module, these data are input into the behavior recognition module.

[0117] The behavior recognition module is the core of the system. It uses a combination of convolutional neural networks (CNN) and recurrent neural networks (RNN) to extract image features and time series features, respectively, and perform behavior recognition. CNN is used to extract image features in video frames, such as the driver's actions and expressions; RNN is used to extract time series features in video frame sequences, such as the duration and frequency of the driver's behavior. The behavior recognition module connects the output features of CNN and RNN, and uses a fully connected layer to map the features to behavior categories, thereby identifying the driver's behavior.

[0118] The evaluation module evaluates the captain's behaviors based on the results of the behavior recognition module and calculates a comprehensive score. The evaluation module takes a weighted average of the captain's behavior scores to obtain a comprehensive score for the captain, and divides the captain into different levels based on the comprehensive score, such as enhanced assistance, qualified, and excellent.

[0119] The report module generates a driver behavior analysis report based on the evaluation results. The report includes the driver's basic information, various behavior analysis results, comprehensive scores, levels, and improvement suggestions. The report module presents the driver's behavior analysis results to fleet managers in a visual way, helping them better understand the driver's work status.

[0120] The management module provides targeted training and management for the drivers based on the driver behavior analysis report. The management module formulates personalized training plans and management programs based on the driver's level and improvement suggestions. For example, it provides driving skills training for drivers with poor driving habits, service awareness training for drivers with poor service levels, psychological counseling for drivers with poor mental outlook, and traffic law training for drivers with poor driving standards.

[0121] The data collection module collects the driver's driving behavior data from the vehicle monitoring equipment, GPS positioning system, vehicle terminal and other equipment, including driving behavior data, voice data and video data. These data can be used to analyze the driver's driving habits, service level, mental outlook, standardized driving and other behaviors.

[0122] Driving behavior data includes: vehicle speed, acceleration, steering angle, braking force, etc., which can be used to analyze the driver's driving habits, such as whether he changes lanes frequently, whether he runs a yellow light, etc.

[0123] Voice data includes: the content of the conversation between the driver and the passengers, which can be used to analyze the driver's service level, such as whether he uses civilized language, whether he actively serves passengers, etc.

[0124] Video data alarm: The driver's actions and expressions during driving can be used to analyze the driver's mental state, standardized driving and other behaviors, such as whether he wears a seat belt and whether he gives way to pedestrians.

[0125] The data preprocessing module uses the minimum-maximum standardization method to perform data standardization. The specific steps are as follows:

[0126] S1. Calculate the maximum and minimum values: For each feature, find the maximum and minimum values ​​in the Internet security information dataset;

[0127] S2. Apply transformation formula: For each feature value x in the Internet security information dataset, use the following formula for transformation:

[0128]

[0129] Among them, x norm is the transformed value;

[0130] S3. Transform the Internet security information dataset: Apply the above transformation to each feature of the entire dataset; after minimum-maximum normalization, all features will be scaled to the range of 0 to 1.

[0131] The feature extraction module extracts features that are helpful for model learning from the preprocessed data; the specific content and operation method are as follows: In this module, the features with high correlation with security events are first screened out through feature selection technology, and then the principal component analysis feature conversion method is used to transform the data to make it more suitable for machine learning model processing; for network traffic data, the feature extraction module may calculate the traffic size, protocol type, and entropy value of the source / destination IP address of the data packet, and use these feature vectors as input for subsequent AI models.

[0132] The behavior recognition module identifies the following:

[0133] Driving habit analysis:

[0134] Identify the driver's driving habits based on driving behavior data, such as:

[0135] Do you give way to pedestrians?

[0136] Doing things unrelated to driving?

[0137] Are you wearing a seat belt?

[0138] Do you drive with one hand?

[0139] Whether the phone is touched, the number of times each behavior is recorded, the frequent locations, the frequent times, etc. Service level analysis:

[0140] Identify the driver's service level based on voice and video data, such as whether he uses civilized language

[0141] Whether to proactively serve passengers

[0142] Indifferent service

[0143] Whether there is quarrel / swearing

[0144] Clean the carriage?

[0145] Whether to dress properly and wear a hat

[0146] Record the number of times, time periods, and other information for each behavior.

[0147] Mental outlook analysis:

[0148] Identify the driver's mental state based on video data, for example:

[0149] Whether to actively send safety alarm

[0150] Whether you are driving while fatigued

[0151] Is the sitting posture improper?

[0152] Is it difficult to concentrate?

[0153] Drink water / eat food?

[0154] Record the number of times, time periods, common locations, etc. of each behavior. Standardized driving analysis:

[0155] Identify the driver's standard driving behavior based on driving behavior data, such as whether he ran a yellow light

[0156] Do you give way to pedestrians?

[0157] Is it an aggressive signal?

[0158] Take the fast lane?

[0159] Whether passengers are picked up or dropped off outside the station

[0160] Whether to press the line

[0161] Whether the station is not in compliance with regulations

[0162] Do you change lanes frequently?

[0163] Whether the zebra crossing does not slow down

[0164] Overtaking in the next lane

[0165] Did you fail to drive in the guide lane?

[0166] Whether the road is flooded and the speed is not reduced

[0167] Record the number of times each behavior occurs, the locations where it often occurs, and other information.

[0168] Other behavioral analyses:

[0169] Identify other behaviors of the driver based on video data, such as:

[0170] Do you listen to the radio?

[0171] Whether the vehicle honks

[0172] Whether to throw objects out of the car window

[0173] Whether the behavior is indecent

[0174] Whether you leave the vehicle without explaining the reason to the passengers

[0175] Did you change lanes on the left without giving a thumbs up?

[0176] Did you turn right without stopping to wait?

[0177] Whether the passenger is not sitting properly and needs to be helped to get out of the car. Record the number, duration, reason, location and other information of each behavior. See the table below for specific analysis content:

[0178]

[0179]

[0180] The behavior recognition module uses 70% of the captain’s behavior data as a training set, 15% of the captain’s behavior data as a validation set, and 15% of the captain’s behavior data as a test set;

[0181] The model training module uses the training set to train the model, optimizes the performance by adjusting the model parameters, and uses the batch gradient descent algorithm to optimize the parameters of the logistic regression model. The iterative update formula is:

[0182] Where θj is the model parameter, α is the learning rate, and J(θ) is the cross entropy loss function, which is calculated as:

[0183]

[0184] Where m is the number of training samples, y(i) is the true label of the i-th sample, and h θ (x (i)) is the prediction result of the model;

[0185] The behavior recognition module then first determines the basic architecture of the model by comparing the performance of different algorithms on the validation set, and then uses hyperparameter tuning technology to find the optimal parameter combination. The model selection and optimization submodule also uses cross-validation methods to evaluate the stability of the model on different data subsets, thereby ensuring that the selected model has good generalization capabilities.

[0186] The behavior recognition module uses the normal driver behavior data set to train the model and determine the K nearest neighbors of each driver behavior data point. Then, for a new driver behavior data point, the average distance between it and the K nearest neighbors is calculated. If this average distance exceeds a preset threshold, the driver behavior data point will be marked as abnormal.

[0187] The calculation formula for anomaly detection is:

[0188]

[0189] Among them: the anomaly score indicates the degree of abnormality of the driver's behavior data point. The higher the score, the greater the possibility of abnormality;

[0190] k represents the number of selected nearest neighbors, that is, the K value in K-NN;

[0191] d(data,neighbor i ): represents the distance between the driver behavior data point and its i-th nearest neighbor; using Euclidean distance as the distance metric, the calculation formula is:

[0192]

[0193] in:

[0194] n represents the number of features of the captain's behavior data points;

[0195] x j,data Represents the value of the current driver behavior data point on the jth feature;

[0196] xj,neighbori represents the value of the i-th nearest neighbor on the j-th feature;

[0197] In this way, the behavior recognition module can automatically learn the distribution characteristics of normal driver behavior data, and identify behaviors that do not conform to normal patterns without prior labeling of abnormal driver behavior data, thereby effectively responding to changes in abnormal driver behavior.

[0198] The evaluation module evaluates the captain's behaviors based on the results of the behavior recognition module and calculates a comprehensive score. The evaluation module takes a weighted average of the captain's behavior scores to obtain a comprehensive score for the captain, and divides the captain into different levels based on the comprehensive score, such as enhanced assistance, qualified, and excellent.

[0199] Behavior evaluation: The evaluation module evaluates the driver's various behaviors, such as whether he gives way to pedestrians, whether he wears a seat belt, whether he uses civilized language, etc. The evaluation module can use different evaluation methods, such as:

[0200] Threshold method: Set a threshold for each behavior, and deduct points if the threshold is exceeded.

[0201] Classification method: Classify behaviors into different categories, such as giving way to pedestrians, not giving way to pedestrians, wearing seat belts, not wearing seat belts, etc., and assign different scores to each category.

[0202] Comprehensive score calculation: The evaluation module takes a weighted average of the scores of the driver's various behaviors to obtain the driver's comprehensive score. The weight can be adjusted according to the actual situation of the fleet, for example, the weight of pedestrian courtesy behavior can be set higher.

[0203] Level division: The evaluation module divides the captains into different levels according to their comprehensive scores, such as enhanced assistance, qualified, and excellent. The level division criteria can be adjusted according to the actual situation of the fleet, for example, a captain with a comprehensive score below 60 points can be classified as enhanced assistance.

[0204] The report module generates a driver behavior analysis report based on the evaluation results. The report includes the driver's basic information, various behavior analysis results, comprehensive scores, levels, and improvement suggestions. The report module presents the driver's behavior analysis results to fleet managers in a visual way, helping them better understand the driver's work status.

[0205] Basic information: including the captain’s name, gender, branch, fleet, vehicle number, etc.

[0206] Behavior analysis results: including the number of times the driver committed each behavior, the score, and the reasons for deductions, etc.

[0207] Comprehensive score: includes the comprehensive score and level of the captain.

[0208] Improvement suggestions: including improvement suggestions for the driver's behavioral issues, such as suggesting that the driver should strengthen his awareness of giving way to pedestrians and regulate his driving behavior.

[0209] Visual presentation: The report module can use charts, tables, etc. to present the driver's behavior analysis results in a visual way. For example, a bar chart can be used to show the number of each behavior of the driver, and a line chart can be used to show the trend of changes in the driver's comprehensive score.

[0210] The management module provides targeted training and management for the drivers based on the driver behavior analysis report. The management module formulates personalized training plans and management programs based on the driver's level and improvement suggestions. For example, it provides driving skills training for drivers with poor driving habits, service awareness training for drivers with poor service levels, psychological counseling for drivers with poor mental outlook, and traffic law training for drivers with poor driving standards.

[0211] Training plan: The management module formulates personalized training plans based on the driver's behavioral problems. For example, for drivers with poor driving habits, driving skills training courses can be organized, such as safe driving training and emergency response training.

[0212] Management plan: The management module formulates different management plans according to the level of the driver. For example, for the driver at the enhanced assistance level, supervision and management can be strengthened, such as regular driving behavior inspections and one-on-one conversations with the driver.

[0213] Incentive mechanism: The management module can establish an incentive mechanism to encourage drivers to develop good driving habits and service awareness, such as rewarding excellent drivers, such as issuing bonuses, promotions, etc.

[0214] In the present invention, the behavior recognition module adopts a deep learning model, a convolutional neural network (CNN) and a recurrent neural network (RNN), which can automatically learn features in the data without manually designing features. This enables the model to more accurately identify the driver's driving habits, service level, mental outlook, standardized driving and other behaviors. The deep learning model can perform end-to-end learning and learn the behavior recognition model directly from the raw data without feature engineering. This simplifies the model training process and improves the model performance. The output features of the CNN and RNN are connected for the recognition module to form a final feature vector for behavior recognition. This multimodal feature fusion method can more comprehensively describe the driver's behavior, thereby improving the accuracy of behavior recognition.

[0215] In the present invention, the output results of the behavior recognition module provide a data basis for the evaluation module and the reporting module. Fleet managers can formulate more scientific management strategies based on these data, such as formulating different training plans and management plans for drivers of different levels, thereby improving management efficiency. The behavior recognition module can help fleet managers understand the behavioral problems of drivers, and thus formulate personalized training plans, such as driving skills training for drivers with poor driving habits, and service awareness training for drivers with poor service levels. This can improve training effectiveness and reduce training costs. The behavior recognition module can identify potential risk factors, such as fatigue driving, irregular driving, etc., and fleet managers can take corresponding measures to control them, such as strengthening the monitoring and management of fatigue driving behavior, thereby reducing risks and improving fleet efficiency. Therefore, the specific algorithm of the internal module of the behavior recognition module can accurately identify the behavior of the driver and provide valuable insights for fleet managers, thereby improving the safety and service quality of the fleet and reducing management costs and risks.

[0216] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the term "comprises" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0217] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An analysis system based on the behavior of the vehicle commander, characterized by: The system includes: a data collection module, a behavior recognition module, an evaluation module, a reporting module, and a management module; The behavior recognition module is internally provided with a data preprocessing module, a feature extraction module, a behavior recognition module and a post-processing module; The data acquisition module is the starting point of the entire driver behavior analysis system. It is responsible for collecting the driver's driving behavior data from the vehicle monitoring equipment, GPS positioning system, and vehicle terminal equipment, including driving behavior data, voice data, and video data; these data are cleaned, enhanced, and annotated by the data preprocessing module and then input into the behavior recognition module; The behavior recognition module is the core of the system. It uses a combination model of convolutional neural network (CNN) and recurrent neural network (RNN) to extract image features and time series features respectively, and perform behavior recognition. CNN is used to extract image features in video frames, including the driver's actions and expressions; RNN is used to extract time series features in video frame sequences, including the duration and frequency of the driver's behavior. The behavior recognition module connects the output features of CNN and RNN, and uses a fully connected layer to map the features to behavior categories, thereby identifying the driver's behavior. The evaluation module evaluates the various behaviors of the driver based on the results of the behavior recognition module and calculates a comprehensive score; the evaluation module performs a weighted average of the scores of the various behaviors of the driver to obtain a comprehensive score of the driver, and classifies the driver into different levels according to the comprehensive score; The report module generates a driver behavior analysis report based on the evaluation results, and the report content includes the driver's basic information, various behavior analysis results, comprehensive scores, levels and improvement suggestions; the report module presents the driver's behavior analysis results to the fleet managers in a visual manner to help them better understand the driver's work status; The management module provides targeted training and management for the driver according to the driver behavior analysis report; the management module formulates personalized training plans and management programs according to the driver's level and improvement suggestions, including driving skills training for the driver with poor driving habits, service awareness training for the driver with poor service level, psychological counseling for the driver with poor mental outlook, and traffic regulations training for the driver with poor driving standards.

2. The analysis system based on the driver's behavior as claimed in claim 1, characterized in that: The data acquisition module collects the driver's driving behavior data from the vehicle-mounted monitoring equipment, GPS positioning system, and vehicle-mounted terminal equipment, including driving behavior data, voice data, and video data.

3. The analysis system based on the driver's behavior as claimed in claim 1, characterized in that: The data preprocessing module uses the minimum-maximum standardization method to perform data standardization processing, and the specific steps are as follows: S1. Calculate the maximum and minimum values: For each feature, find the maximum and minimum values ​​in the Internet security information dataset; S2. Apply transformation formula: For each feature value x in the Internet security information dataset, use the following formula for transformation: Among them, x norm is the transformed value; S3. Transform the Internet security information dataset: Apply the above transformation to each feature of the entire dataset; after minimum-maximum normalization, all features will be scaled to the range of 0 to 1.

4. The analysis system based on the driver's behavior as claimed in claim 1, characterized in that: The feature extraction module extracts features that are helpful for model learning from the preprocessed data. It first uses feature selection technology to screen out features that are highly correlated with security events, and then uses the principal component analysis feature conversion method to convert the data to make it more suitable for machine learning model processing; for network traffic data, the feature extraction module calculates the traffic size, protocol type, and entropy value of the source / destination IP address of the data packet, and uses these feature vectors as input for subsequent AI models.

5. The analysis system based on the driver's behavior as claimed in claim 1, characterized in that: The identification content of the behavior identification module includes: Driving habit analysis: Identify the driver’s driving habits based on driving behavior data, including: Do you give way to pedestrians? Doing things unrelated to driving? Are you wearing a seat belt? Do you drive with one hand? Whether to touch the phone Record the number of times, common locations, and common times of each behavior; Service Level Analysis: Identify the captain’s service level based on voice and video data, including: Whether to use civilized language Whether to proactively serve passengers Indifferent service Whether there is quarrel / swearing Clean the carriage? Whether to dress properly and wear a hat Record the number and time period of each behavior; Mental outlook analysis: Identify the driver's mental state based on video data, including: Whether to actively send safety alarm Whether you are driving while fatigued Is the sitting posture improper? Is it difficult to concentrate? Drink water / eat food? Record the number, time period, and frequent locations of each behavior; standardize driving analysis: Identify the driver's standard driving behavior based on driving behavior data, including: whether to run a yellow light Do you give way to pedestrians? Is it an aggressive signal? Take the fast lane? Whether passengers are picked up or dropped off outside the station Whether to press the line Whether the station is not in compliance with regulations Do you change lanes frequently? Whether the pedestrian crossing does not slow down Overtaking in the next lane Did you fail to drive in the guide lane? Whether the road surface is flooded and the speed is not reduced Record the number of times each behavior occurs and the locations where it is often performed; Other behavioral analyses: Other behaviors of the driver can be identified based on video data, including: Do you listen to the radio? Whether the vehicle honks Whether to throw objects out of the car window Whether the behavior is indecent Whether you leave the vehicle without explaining the reason to the passengers Did you change lanes on the left without giving a thumbs up? Did you turn right without stopping to wait? Is the passenger not sitting properly and holding the car steady? Record the number, duration, reason, and location of each behavior.

6. The analysis system based on the driver's behavior as claimed in claim 1, characterized in that: The behavior recognition module uses 70% of the driver behavior data as a training set, 15% of the driver behavior data as a validation set, and 15% of the driver behavior data as a test set; The model training module uses the training set to train the model, optimizes the performance by adjusting the model parameters, and uses the batch gradient descent algorithm to optimize the parameters of the logistic regression model. The iterative update formula is: Where θj is the model parameter, α is the learning rate, and J(θ) is the cross entropy loss function, which is calculated as: Where m is the number of training samples, y(i) is the true label of the i-th sample, and h θ (x (i)) is the prediction result of the model; The behavior recognition module then first determines the basic architecture of the model by comparing the performance of different algorithms on the validation set, and then uses hyperparameter tuning technology to find the optimal parameter combination. The model selection and optimization submodule also uses cross-validation methods to evaluate the stability of the model on different data subsets, thereby ensuring that the selected model has good generalization capabilities.

7. The analysis system based on the driver's behavior as claimed in claim 1, characterized in that: The behavior recognition module uses the normal driver behavior data set to train the model and determine the K nearest neighbors of each driver behavior data point. Then, for a new driver behavior data point, the average distance between it and the K nearest neighbors is calculated. If the average distance exceeds a preset threshold, the driver behavior data point will be marked as abnormal. The calculation formula for anomaly detection is: Among them: the anomaly score indicates the degree of abnormality of the driver's behavior data point. The higher the score, the greater the possibility of abnormality; k represents the number of selected nearest neighbors, that is, the K value in K-NN; d(data,neighbor i ): represents the distance between the driver’s behavior data point and its i-th nearest neighbor; Using Euclidean distance as the distance metric, the calculation formula is: in: n represents the number of features of the captain's behavior data points; x j,data Represents the value of the current driver behavior data point on the jth feature; xj,neighbori represents the value of the i-th nearest neighbor on the j-th feature; In this way, the behavior recognition module can automatically learn the distribution characteristics of normal driver behavior data, and identify behaviors that do not conform to normal patterns without prior labeling of abnormal driver behavior data, thereby effectively responding to changes in abnormal driver behavior.

8. The analysis system based on the driver's behavior as claimed in claim 1, characterized in that: The evaluation module performs a weighted average of the scores of the various behaviors of the driver to obtain a comprehensive score of the driver, and classifies the driver into different levels according to the comprehensive score, including enhanced assistance, qualified, and excellent.

9. The analysis system based on the driver's behavior as claimed in claim 1, characterized in that: The report module generates a driver behavior analysis report based on the evaluation results, and the report content includes the driver's basic information, various behavior analysis results, comprehensive scores, levels and improvement suggestions; The reporting module presents the driver's behavior analysis results to fleet managers in a visual manner, helping them better understand the driver's work status.

10. The analysis system based on the driver's behavior as claimed in claim 1, characterized in that: The management module provides targeted training and management to the driver according to the driver behavior analysis report; The management module formulates personalized training plans and management programs based on the captain's level and improvement suggestions, including driving skills training for captains with poor driving habits and service Provide service awareness training to drivers with poor skills and psychological counseling to drivers with poor mental outlook. Provide traffic law training to drivers with poor driving standards.

Images