A truck warning status recognition method based on convolutional neural networks

Abstract

This invention relates to the field of traffic safety application technology and discloses a truck warning status recognition method based on convolutional neural networks, comprising the following steps: processing GPS trajectory data; analyzing truck warning data features; and determining the optimal network structure. This truck warning status recognition method based on convolutional neural networks establishes a mapping relationship between truck warning status and trajectory segment features by constructing a kinematic feature index system. It proposes a truck warning status recognition technology to provide technical support for identifying safety risks through abnormal trajectories, reducing the probability of accidents, improving truck operation safety and driving efficiency, and ensuring road traffic safety.

Description

Technical Field

[0001] This invention relates to the field of traffic safety application technology, specifically to a method for recognizing truck warning status based on convolutional neural networks. Background Technology

[0002] With the rapid development of the global economy and urbanization, truck transportation has become an important component of the modern logistics system. Against the backdrop of a continuously recovering economic environment, road freight turnover is showing an upward trend. However, safety issues in truck transportation are becoming increasingly prominent. Compared to other types of vehicle accidents, the consequences of freight vehicle accidents are more severe.

[0003] Freight traffic accidents not only disrupt traffic order but also cause enormous economic and incalculable social losses. Therefore, risk warning and control during truck operations have become crucial aspects of road management for road administrators and operators. The factors contributing to the frequent occurrence of truck accidents are complex and diverse, with the driver's condition and the vehicle's operational status directly affecting truck safety. Therefore, real-time monitoring and early warning of truck driver conditions and vehicle operating conditions are of paramount importance.

[0004] With the development of advanced technologies such as the Internet of Things, artificial intelligence, and sensor technology, real-time monitoring and analysis of truck status has become possible. However, many current research methods require manual design to extract features related to truck behavior, which is easily influenced by the researcher's subjective opinion and cannot be optimized by algorithms. This results in features that are not objective and accurate enough, failing to fully capture the deep features and complex correlation patterns of the data; or they require a lot of manpower and time to annotate truck warning data. To address the shortcomings of current research, the relationship between truck trajectory features and warning risks can be established to achieve accurate judgment of driver status, vehicle operation, etc., thereby enabling timely warnings of potential safety risks.

[0005] In view of this, we propose a truck warning status recognition method based on convolutional neural networks. Summary of the Invention

[0006] The purpose of this invention is to provide a truck warning status recognition method based on convolutional neural networks to solve the problems mentioned in the background art.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a truck warning status recognition method based on convolutional neural networks, the method comprising the following steps:

[0008] S1. Processing of GPS trajectory data;

[0009] S11, Handling outlier data;

[0010] S12, Coordinate Transformation;

[0011] S2, Truck Early Warning Data Feature Analysis;

[0012] S21. Extraction of truck kinematic trajectory;

[0013] S22. Truck warning trajectory feature analysis: Visualize and statistically analyze truck trajectory data to identify the distribution range and characteristics of truck trajectory points under different states such as fatigue driving warning, overspeed warning, rollover warning, and collision warning, as well as the speed status of trucks at different locations. Hourly statistics of trajectory points within a week are performed according to different weeks to calculate the duration of warning trajectory segments.

[0014] S23. Truck warning trajectory segmentation: Based on the characteristic study of warning trajectory segments for speeding, fatigued driving, rollover, and collision, the trajectory is segmented using sample short-stroke segmentation based on changes in vehicle kinematic characteristics. Trajectory length selection values ​​are determined, and trajectory filtering is performed.

[0015] S24. Sample parameter completion: Complete the information of time, distance, velocity, acceleration, jerk, angle difference, and angle difference change rate on the preprocessed trajectory data.

[0016] S3. Truck warning status recognition based on convolutional neural network;

[0017] S31. Initialize network layers and parameters;

[0018] S32. Construct a neural network, using a combination of components such as input layer, convolutional layer, batch normalization layer, activation function, max pooling layer, and fully connected layer to design the neural network structure; employ feature fusion and Dropout techniques for fully connected layer design.

[0019] S33. Model training: The raw data is converted into PyTorch tensors for interaction with the neural network. Then, the training data is iteratively processed. The model's predictions are calculated via forward propagation, and the loss value is calculated based on the predictions and the true labels. Next, the gradient is calculated using the backpropagation algorithm, and the network parameters are updated. This process is repeated until the model converges to its optimal parameters.

[0020] S34. Model Testing: Compare the model's prediction results with the true labels to calculate test accuracy. Calculate metrics such as confusion matrix, accuracy, and F1 score to comprehensively evaluate the model's performance. These metrics intuitively reflect the model's performance in different aspects, providing important reference for subsequent model optimization.

[0021] S4. Determine the optimal network structure.

[0022] Optionally, step S11 includes the following steps: acquiring GPS data, determining the threshold for the normal operating speed of trucks, and removing abnormal trajectory point data with an average speed greater than the threshold. Simultaneously, abnormal latitude and longitude coordinate data that are outside the study area, as well as duplicate data, are also filtered out.

[0023] S12 includes the following steps: In order to ensure the consistency and comparability of the data, coordinate transformation is performed using the WGS-84 coordinate system as a unified standard, making the visualization of trajectory data on the map more convenient and intuitive, and facilitating effective analysis and application.

[0024] Optionally, S21 includes:

[0025] The system calculates the truck's operating speed, acceleration, and steering angle and rate of change of steering angle; it detects dangerous behaviors such as sudden braking or acceleration by analyzing instantaneous speed and acceleration changes; it determines whether the truck is turning by analyzing the steering angle and rate of change of steering angle; and it characterizes and analyzes dangerous behaviors such as speeding, fatigued driving, rollover, or collision based on the calculated data. The specific calculations are as follows:

[0026] The Haversine formula is used to calculate the geographical distance between two adjacent points:

[0027] a = sin 2 (Δlat / 2)+cos(lat i )*cos(lat i+1 )*sin 2 (Δlon / 2) (1)

[0028] c = 2 * asin(sqrt(a)) (2)

[0029] d = R * c (3)

[0030] Where d represents the spherical distance between the two points, R represents the average radius of the Earth, and lat i and lat i+1 Δlat represents the latitude of two adjacent points, Δlat = lat i+1 -lat i Representing the difference in latitude, Δlon = lon i+1 -lon i Represents the difference in longitude;

[0031] Acceleration and jerk calculation:

[0032] a i =Δv / Δt (4)

[0033] ji =Δa / Δt (5)

[0034] Calculation of change in direction angle and rate of change:

[0035] bc i =Δθ=θ i+1 -θ i (6)

[0036] ba i =Δθ / Δt=θ i+1 -θ i / Δt (7)

[0037] Where Δv represents the change in velocity between adjacent points i and i+1, Δa represents the change in acceleration between adjacent points i and i+1, and Δt represents the change in time.

[0038] Optionally, in step S23, the trajectory points with an average velocity of 0 are processed in different cases, and the specific division steps are as follows:

[0039] S231. Divide the trajectory into segments by performing short-stroke segmentation;

[0040] S232. Calculate the basic characteristics of each trajectory point and filter out the trajectory data with a length not less than a certain value that are in one of the four warning states.

[0041] S233. Sort and number the trajectory points according to the license plate number and time, and make preliminary divisions for different time periods of different vehicles, taking the continuous journey of each vehicle as the short journey division unit.

[0042] S234. For trajectory points with an average velocity of 0, handle them in two cases: for points where adjacent trajectory points all have non-zero velocities and an average velocity of 0, set them as breakpoints. For long, continuous segments with zero velocity, delete them and divide the trajectory into segments at the next non-zero velocity point, updating the numbering accordingly.

[0043] S235. After the short-stroke trajectory segment is divided, extract the trajectory segment with a length not less than a certain value.

[0044] S236. Trajectory segment integration and encoding update.

[0045] Optionally, step S32 further includes the following specific construction steps:

[0046] S321. Transform the trajectories in the same segment into multiple dimensions. The input matrix is ​​in the form of formula (8), where the rows of the matrix represent a single feature, the columns represent all features of the sampling point at the current time, the width of the input matrix is ​​the dimension of the input feature, which is consistent with the dimension of the model input layer, and the length m of the input matrix represents the number of trajectory points contained in a single sample.

[0047]

[0048] S322. Make the original trajectory samples have the same length by filling or truncating, and take a sliding window M=50 (i.e. 50 trajectory points) trajectory segment; truncate the segments with a length of more than 50, and fill the segments with less than 50 with the value of the last trajectory point;

[0049] S323. An embedding layer fusion technique is introduced to fuse different features of truck operation data; random initialization is adopted, and the size of the embedding layer (i.e., the dimension of each category vector) is a hyperparameter; the embedding vector of each classification feature is obtained, and they are fused with other types of features (such as continuous features, other embedding vectors, etc.) through weighted summation, and the fused features are used as input;

[0050] S324. A two-dimensional convolutional neural network is used, which treats the matrix of multi-dimensional features as image input, thus taking into account the correlation between different feature columns simultaneously. Its convolution formula is as follows:

[0051]

[0052] S325. Add residual connections to the network structure. Residual connections connect the input directly to the deep output of the network.

[0053] S326. Use a batch normalization (BN) layer to standardize the data. Let the input data be {X1, X2, ..., Xn}.

[0054] Calculate the mean μ of the n input sample data using formula (10):

[0055]

[0056] The variance (sigma) σ of the input sample data is calculated using formula (11). 2 :

[0057]

[0058] Use formula (12) to remove the mean from the input sample data and divide by the standard deviation:

[0059]

[0060] Introducing parameters gama and lambda to scale and shift the standardized data prevents normally distributed standardized data from being distributed on both sides of the sigmoid function and the normalized data from being concentrated in the approximately linear part of the sigmoid function, which would affect the nonlinearity.

[0061]

[0062] S327. A pooling layer is periodically inserted between convolutional layers. This invention uses max pooling, selecting the maximum value in each pooling window. The calculation formula is as follows:

[0063]

[0064] Among them, z i,k z is the output of the current layer. i-1,k+m This refers to the m-th neuron in the previous layer;

[0065] S328. The fully connected layer is designed using feature fusion and Dropout techniques. The output of the pooling layer is flattened. The flattening operation stretches the feature map into a one-dimensional vector. The fully connected layer receives the flattened input. Each neuron is connected to each element of the input, and a weighted sum is performed with a bias added. The fully connected layer maps the final output to a linearly separable space and works with the output layer for classification.

[0066] S329. Finally, the data enters the output layer, which uses a softmax classifier for classification, as shown in the following formula:

[0067]

[0068] Ultimately, each position receives a probability value, and the value of the position with the highest probability is used as the recognition result.

[0069] Optionally, S34 further includes:

[0070] The specific calculation is as follows: Confusion Matrix: The confusion matrix is ​​a two-dimensional matrix that shows the model's predictions on different categories. For multi-class classification problems, the concept of the confusion matrix is ​​expanded.

[0071] The formula (16) for calculating accuracy is:

[0072]

[0073] The formula (17) for calculating precision is:

[0074]

[0075] The recall rate is calculated using the formula (18):

[0076]

[0077] The formula (19) for calculating the F1 score is as follows:

[0078]

[0079] The confusion matrix will contain true positives (TP), false positives (FP), and false negatives (FN) for each class.

[0080] Optionally, S4 includes:

[0081] The performance of different network configurations is quantitatively evaluated, and key hyperparameter configurations are extracted. These hyperparameters include layer patterns (i.e., combinations of convolutional layers, pooling layers, batch normalization layers, etc.), the presence or absence of layers (i.e., whether specific network layers are included), the convolutional depth of the CNN (i.e., the number of convolutional layers), and the number of filters per layer. By changing the hyperparameter configurations, the most suitable network structure is ultimately found for truck warning status recognition.

[0082] Compared with existing technologies, this invention provides a truck warning status recognition method based on convolutional neural networks, which has the following beneficial effects:

[0083] This truck warning status recognition method based on convolutional neural networks establishes a mapping relationship between truck warning status and trajectory segment features by constructing a kinematic feature index system. It proposes a truck warning status recognition technology to provide technical support for identifying safety risks through abnormal trajectories, reducing the probability of accidents, improving the safety and efficiency of truck operation, and ensuring road traffic safety. Attached Figure Description

[0084] Figure 1 This is an overall flowchart of the present invention;

[0085] Figure 2 This is an example diagram of the orientation angle of the present invention;

[0086] Figure 3 This is a time distribution diagram of the early warning trajectory points of the present invention;

[0087] Figure 4 This is a diagram illustrating the trajectory division of the present invention;

[0088] Figure 5 This is a flowchart of the trajectory drawing process of the present invention;

[0089] Figure 6 This is an example diagram of two-dimensional convolution in this invention;

[0090] Figure 7 This is a schematic diagram of the residual connection of the present invention;

[0091] Figure 8This is a schematic diagram of the maximum pooling process of the present invention;

[0092] Figure 9 This is a structural diagram of the truck early warning status prediction algorithm of the present invention. Detailed Implementation

[0093] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0094] like Figures 1-9 As shown, the present invention provides a technical solution: a truck warning status recognition method based on convolutional neural networks, comprising the following steps:

[0095] S1, Processing of GPS trajectory data

[0096] S11, Handling outliers

[0097] GPS data was acquired, with trajectory data sampling intervals typically around 30 seconds. Each trajectory point contains 31 fields; seven relevant fields were selected for study, including vehicle number, longitude, latitude, time, instantaneous speed, angle with true north, and warning signs. The data format obtained from the sampling is shown in the table below:

[0098]

[0099] Examples of actual trajectory data are shown in Table 2:

[0100] license plate number longitude latitude time included angle speed km / h Alarm Tag Shaanxi AC***1 109.09880 34.387989 11 / 2 13：36：37 188 14.9 2 Shaanxi AC***1 109.09854 34.386863 11 / 2 13：37：07 191 11.1 2 Shaanxi AC***1 109.09830 34.385747 11 / 2 13：37：37 189 18.6 2 Shaanxi AC***1 109.09802 34.384354 11 / 2 13：38：07 190 20.5 2 Shaanxi AC***1 109.09782 34.383368 11 / 2 13：38：37 176 16.6 2

[0101] Considering the possibility of speeding by trucks, abnormal trajectory point data with an average speed greater than 150 km / h were removed. Additionally, abnormal latitude and longitude coordinate data whose values ​​were outside the study area were also deleted. When GPS signals are interfered with or blocked, location data may be recorded repeatedly. This duplicate abnormal data was then deleted.

[0102] S12, Coordinate Transformation

[0103] Using the WGS-84 coordinate system as a unified standard, the original truck GPS trajectory data was converted from the GCJ-02 coordinate system (x, y) to the WGS-84 coordinate system. The calculation method for (x', y') in the coordinate system is as follows:

[0104] x'=2x-(x+dLon) (1)

[0105] y'=2y-(y+dLat) (2)

[0106]

[0107] Where, e = 6378245, e′ = 0.0066934216229659;

[0108] S2. Truck early warning data feature analysis:

[0109] S21. Extraction of truck kinematic trajectory

[0110] GPS latitude and longitude can be used to calculate the distance a vehicle travels per unit time, while the vehicle's instantaneous speed and its angle with true north can characterize the truck's current motion trend. Acceleration reflects the change in the vehicle's speed per unit time, representing the vehicle's acceleration and deceleration. The angle with true north and its rate of change reflect the rate of change of the vehicle's direction per unit time, indicating the vehicle's steering. The specific steps for calculating the truck's speed, acceleration, and the truck's heading angle and rate of change of heading angle are as follows:

[0111] The Haversine formula is used to calculate the geographical distance between two adjacent points:

[0112] a = sin 2 (Δlat / 2)+cos(lat i )*cos(lat i+1 )*sin 2 (Δlon / 2) (6)

[0113] c = 2 * asin(sqrt(a)) (7)

[0114] d = R * c (8)

[0115] Calculate acceleration and jerk:

[0116] a i =Δv / Δt (9)

[0117] j i =Δa / Δt (10)

[0118] Calculate the change in direction angle and the rate of change:

[0119] bc i =Δθ=θ i+1 -θ i (11)

[0120] ba i =Δθ / Δt=θ i+1 -θi / Δt (12)

[0121] Examples of specific orientation angles are as follows: Figure 2 As shown;

[0122] S22, Truck Warning Trajectory Feature Analysis

[0123] The trajectory data of trucks is visualized and statistically analyzed to generate a distribution map of warning trajectory points and a heatmap weighted by the speed of these points. The distribution map identifies the range and characteristics of truck trajectory points under different warning states: fatigue driving warning, speeding warning, rollover warning, and collision warning. The heatmap, weighted by the speed of the warning trajectory points, visually displays the speed of the warning vehicles at different locations, helping managers identify high-risk areas and the speed status of trucks at different locations. The time of the trajectory points within a week is statistically analyzed hourly according to different weeks to obtain an hourly distribution heatmap of the warning trajectory points, identifying periods when the trajectory warning data appears significantly.

[0124] The characteristics of the warning trajectory under visualization and statistical analysis are shown in the table below:

[0125]

[0126] Time distribution of warning trajectory points as follows Figure 3 As shown;

[0127] S23, Truck Warning Trajectory Segment Division

[0128] Based on the characteristic study of trajectory segments for speeding, fatigued driving, rollover, and collision warnings, a short-stroke segmentation method based on changes in vehicle kinematic characteristics is used to segment the trajectories. Trajectory length selection values ​​are determined, and trajectory selection is performed. Trajectory points with an average speed of 0 are handled differently depending on the case; the specific segmentation steps are as follows:

[0129] S231. The trajectory is segmented using short-stroke segmentation based on changes in vehicle kinematic characteristics. The short stroke mainly includes four driving characteristics: acceleration, deceleration, constant speed, and idle speed. An acceleration of 0.15 m / s² is used. 2 As a threshold, the acceleration segment is defined as a>=0.15m / s. 2 Meanwhile, V≠0, and the deceleration segment is a<=-0.15m / s. 2 Meanwhile, V≠0, and the uniform velocity segment is |a|<0.15m / s 2 At the same time, V≠0, the idle speed range is when a=0 and V=0 and the engine runs continuously;

[0130] S232. Define the length of the trajectory segment as the number of trajectory points contained in the trajectory segment. Based on the example, use 10 as the threshold for the length of the trajectory points, calculate the basic characteristics of each trajectory point, and filter out the trajectory data with a length of not less than 10 that are in the four warning states.

[0131] S233. Sort and number the trajectory points according to the license plate number and time, and make preliminary divisions for different time periods of different vehicles, taking the continuous journey of each vehicle as the short journey division unit.

[0132] S234. If a truck is in the following situations: ① waiting at a signalized intersection at a red light. ② on urban roads with high traffic density, the truck may need to briefly stop and wait in congested traffic. ③ loading and unloading goods, adjusting the position of goods, etc., require brief stops to complete necessary operations. ④ in specific traffic control areas or speed-restricted areas, it may need to briefly slow down or stop due to traffic congestion or speed limits. Trucks may make brief stops to comply with traffic regulations, ensure traffic safety, or complete cargo transportation tasks. For points where adjacent trajectory points all have non-zero speeds and the average speed is 0, a breakpoint is set. For long continuous segments with zero speed, delete them and divide the trajectory segment at the next non-zero speed point, updating the numbering.

[0133] S235. After the short-stroke trajectory segment is divided, extract trajectory segments with a length of not less than ten, which is defined as...

[0134] The function is used to extract all trajectory segments with the same number at least ten.

[0135] S236. Trajectory segment integration and encoding update;

[0136] The final number of valid short-distance trajectory points extracted was 617,362, with 23,410 trajectory segments. The final sample size after trajectory segmentation is shown in the table below:

[0137] Warning situation Collision warning Speeding warning Side rollover warning Fatigue driving warning Number of trajectory segments 3480 4274 7733 7923 Number of trajectory points (number) 106102 133687 116319 261254

[0138] See trajectory division details Figure 4 For the specific trajectory division process, please refer to [link / reference]. Figure 5 ;

[0139] S24, Sample Parameter Completion

[0140] The preprocessed trajectory data is supplemented with information on time, distance, velocity, acceleration, jerk, angle difference, and angle difference transformation rate. The specific parameters of the final sample points after trajectory segmentation are shown in Table 5.

[0141]

[0142]

[0143] S3. Truck warning status recognition based on convolutional neural network:

[0144] S31. Initialize network layers and parameters

[0145] S311. Develop the architecture of a neural network model, which includes multiple components such as convolutional layers, batch normalization layers, activation functions, and fully connected layers.

[0146] S312. Define the forward propagation process, meaning that information can only be propagated in one direction, and there is no round-trip loop. The output of the previous layer is used as the input of the next layer, and the output of the next layer is calculated, continuing until the output layer is reached.

[0147] S313. Add activation functions. The activation functions must satisfy the following properties: 1. Nonlinearity; 2. Differentiability; 3. Monotonically increasing; 4. Bounded output range. The outputs of the convolutional layers all use the ReLU activation function to increase the model's nonlinear expressiveness.

[0148] S314. The cross-entropy loss function is chosen, as it effectively measures the difference between the model's predictions and the true labels, helping the model learn the correct classification decision. Simultaneously, the Adam optimizer is selected as the model's optimization algorithm. Adam combines the advantages of momentum gradient descent and RMSprop algorithms, adaptively adjusting the learning rate to accelerate the model's training process.

[0149] S32. Constructing a neural network

[0150] A neural network structure is designed using a combination of components such as input layer, convolutional layer, batch normalization layer, activation function, max pooling layer, and fully connected layer. Residual connections are added to the network structure to address the vanishing and exploding gradient problems that occur during deep neural network training. A fusion technique in the embedding layer is introduced to fuse different features of the truck operation data, further enriching the model's input information and improving prediction accuracy. Feature fusion and Dropout techniques are used in the fully connected layer design. The specific construction steps are as follows:

[0151] S321. The trajectories in the same segment are divided into multiple dimensions. The feature parameters include velocity, acceleration, jerk, distance, angle with due north, angle difference, angle difference transformation rate, and the square of the feature parameters that are subtracted. The input matrix is ​​in the form of formula (8), where the rows of the matrix represent a single feature, the columns represent all features of the sampling point at the current time, the width of the input matrix is ​​the dimension of the input feature, which is consistent with the dimension of the model input layer, and the length m of the input matrix represents the number of trajectory points contained in a single sample.

[0152]

[0153] S322. First, determine the number of trajectory points to be divided, i.e., the window size, set to 50. Then initialize two pointers, start and end, which represent the start and end positions of the window, respectively. Initially, both start and end point to the start position of the trajectory segment. end moves forward until the window contains the specified number of trajectory points. During the movement, record the start and end positions of each window. When the window reaches the boundary of the trajectory segment's end, i.e., the last window may not contain enough trajectory points to fill the window size, select the value of the last trajectory point in the current window to fill it, until its length reaches 50.

[0154] S323. An embedding layer fusion technique is introduced to fuse different features of truck operation data, enriching the model's input information and improving prediction accuracy. The embedding layer can be initialized randomly. The size of the embedding layer is a hyperparameter, with an optimal value of 30. Once the embedding vector for each categorical feature is obtained, it can be fused with other features using methods such as weighted summation. The embedding layer settings are shown in the table below:

[0155]

[0156]

[0157] S324. A two-dimensional convolutional neural network is used, which treats the matrix of multi-dimensional features as image input, thus taking into account the correlation between different feature columns simultaneously. Its convolution formula is as follows:

[0158]

[0159] Two-dimensional convolution example as follows Figure 6 As shown;

[0160] S325. Add residual connections to the network structure. Residual connections directly connect the input to the deep output of the network. In this example, the residual block consists of several convolutional layers (or other types of layers) and a "short-circuit" connection. This "short-circuit" connection adds the block's input (or some transformed input) to the block's output and then passes it to the ReLU activation function. Figure 7 This is a schematic diagram of the residual connection;

[0161] S326. Use a batch normalization (BN) layer to standardize the data. Let the input data be {X1, X2, ..., Xn}.

[0162] Calculate the mean μ of the n input sample data using formula (15):

[0163]

[0164] The variance (sigma) σ of the input sample data is calculated using formula (16). 2 :

[0165]

[0166] Use formula (17) to remove the mean from the input sample data and divide by the standard deviation:

[0167]

[0168] Introducing parameters gama and lambda to scale and shift the standardized data prevents normally distributed standardized data from being distributed on both sides of the sigmoid function and the normalized data from being concentrated in the approximately linear part of the sigmoid function, which would affect the nonlinearity.

[0169]

[0170] Normalization layers normalize the outputs of intermediate layers in the network, which speeds up model training time, improves the model's generalization ability, and helps prevent problems such as gradient vanishing or gradient exploding.

[0171] S327. Periodically insert a pooling layer between convolutional layers, using max pooling, and selecting the maximum value within each pooling window. The max pooling process is as follows: Figure 8 As shown, the calculation formula is as follows:

[0172]

[0173] Among them, z i,k z is the output of the current layer. i-1,k+m This refers to the m-th neuron in the previous layer;

[0174] S328. The fully connected layer design employs feature fusion and Dropout techniques. Feature fusion combines features from different levels, fully utilizing the information extracted by the model at different levels. In the Dropout layer, the P-value is initially set to 0.5, meaning that 50% of the neurons will be randomly disconnected in each training batch. Early stopping is used to automatically select the optimal number of training epochs.

[0175] The pooling layer output is flattened, stretching the feature map into a one-dimensional vector. The fully connected layer receives the flattened input, with each neuron connected to each element of the input. Weighted summation is performed, and a bias is added. The fully connected layer maps the final output to a linearly separable space, working with the output layer for classification.

[0176] S329. Finally, the output layer is entered. The output layer uses a softmax classifier for classification. The formula is shown in (20). Finally, each position (a total of four positions, representing four warning states) has a probability value. This probability value is the probability that the input trajectory segment is in the current warning state. Finally, the value of the position with the highest probability is used as the recognition result.

[0177]

[0178] S33, Model Training

[0179] S331. During the model training phase, the cross-entropy loss function was chosen. This function effectively measures the difference between the model's predicted results and the true labels, helping the model learn the correct classification decisions. Simultaneously, the Adam optimizer was selected as the model's optimization algorithm. Adam combines the advantages of momentum gradient descent and RMSprop algorithms, adaptively adjusting the learning rate and accelerating the model training process.

[0180] S332. During training, multiple hyperparameters are set to control the model's training behavior. The outputs of convolutional layers all use the ReLU activation function to increase the model's non-linear expressive power. The maximum number of training iterations is set to 100, and the sample size for each batch is set to 64. Furthermore, Dropout technology is introduced in the fully connected layers to prevent overfitting by randomly disconnecting a portion of neuron connections. In the Dropout layer, the initial P-value is set to 0.5, meaning that 50% of neurons will be randomly disconnected in each training batch. To further improve model performance, early stopping is used to automatically select the optimal number of training epochs. This technique monitors performance metrics (such as accuracy) on the validation set to determine if the model is overfitting. When the performance metrics on the validation set do not improve for several consecutive epochs, training is stopped, and the parameters of the best-performing model are saved.

[0181] S34, Model Testing

[0182] After model training, the model was evaluated using an independent test set. Performance metrics (such as accuracy, confusion matrix, and F1 score) on the test set were calculated to comprehensively evaluate the model's generalization ability and prediction accuracy. The specific calculations are as follows:

[0183] Confusion Matrix: A confusion matrix is ​​a two-dimensional matrix that shows the model's predictions across different classes. This example involves four warning states, constituting a multi-class classification problem, thus expanding the concept of the confusion matrix. The confusion matrix will contain True Positives (TP), False Positives (FP), and False Negatives (FN) for each class. Table 7 shows an example confusion matrix for a four-class problem: rows represent the true classes, and columns represent the predicted classes;

[0184] Predicted as Category 1 Predicted as Category 2 Predicted as Category 3 Predicted as Category 4 Actually, it is category 1. TP FN FN FN Actually Category 2 FP TP FN FN Actually, it is category 3. FP FP TP FN Actually, it is category 4. FP FP FP TP

[0185] True Positive Case (TP): The number of positive class samples correctly predicted by the model;

[0186] False positives (FP): The number of negative class samples that the model incorrectly predicts as positive. FP represents a false positive by the model, where negative class samples are incorrectly classified as positive.

[0187] False Negatives (FN): The number of positive class samples that the model incorrectly predicts as negative. FN represents a false positive, where a positive class sample is incorrectly classified as a negative class.

[0188] The elements on the diagonal represent the number of samples correctly predicted by the model, while the elements off-diagonal represent the number of incorrect predictions made by the model in that category.

[0189] Accuracy:

[0190]

[0191] Precision:

[0192]

[0193] Recall:

[0194]

[0195] The F1 score is:

[0196]

[0197] S4. Determine the optimal network structure:

[0198] The performance of different network configurations was quantitatively evaluated, and key hyperparameter configurations were extracted. These hyperparameters included layer patterns (i.e., combinations of convolutional layers, pooling layers, Batch Normalization layers, etc.), the presence or absence of layers (i.e., whether specific network layers are included), the convolutional depth of the CNN (i.e., the number of convolutional layers), and the number of filters per layer. An example was created by modifying the hyperparameter configurations of seven network models; the results are shown in Table 8.

[0199]

[0200]

[0201] Based on overall accuracy, CNN-D is the optimal network structure. The structure of the truck warning status prediction algorithm is shown below. Figure 9 .

[0202] Therefore, the above-mentioned truck warning status recognition method based on convolutional neural network can identify truck speeding, fatigue driving, rollover and collision warnings, which has high application value in the field of traffic safety.

[0203] The present invention has been described in detail above. However, modifications or improvements can be made to it, which will be obvious to those skilled in the art. Therefore, any modifications or improvements that do not depart from the spirit of the present invention are within the scope of protection of the present invention.

Claims

1. A method for recognizing truck warning status based on convolutional neural networks, characterized in that: The method includes the following steps: S1. Processing of GPS trajectory data; S11, Handling outlier data; S12, Coordinate Transformation; S2, Truck Early Warning Data Feature Analysis; S21. Extraction of truck kinematic trajectory; S22. Truck warning trajectory feature analysis: Visualize and statistically analyze the truck trajectory data to identify the distribution range and characteristics of truck trajectory points under different states such as fatigue driving warning, overspeed warning, rollover warning, and collision warning, as well as the speed status of the truck at different locations. The time of trajectory points within a week is statistically analyzed hourly according to different weeks to calculate the duration of the warning trajectory segment; S23. Truck warning trajectory segment division: Based on the characteristic study of warning trajectory segments for speeding, fatigue driving, rollover, and collision, the trajectory is segmented using sample short-stroke segmentation based on changes in vehicle kinematic characteristics; trajectory length screening values ​​are determined, and trajectory screening is performed. S24. Sample parameter completion: Complete the information of time, distance, velocity, acceleration, jerk, angle difference, and angle difference change rate on the preprocessed trajectory data. S3. Truck warning status recognition based on convolutional neural network S31. Initialize network layers and parameters; S32. Construct a neural network by designing a neural network structure using a combination of input layer, convolutional layer, batch normalization layer, activation function, max pooling layer, and fully connected layer components; and use feature fusion and Dropout techniques for fully connected layer design. S33. Model training: Convert the raw data into PyTorch tensors to interact with the neural network; then, iteratively process the training data, calculate the model's prediction results through forward propagation, and calculate the loss value based on the prediction results and the true labels; use the backpropagation algorithm to calculate the gradient and update the network parameters; repeat this process until the model converges to the optimal parameters. S34. Model testing: Compare the model's prediction results with the true labels to calculate the test accuracy; calculate the confusion matrix, accuracy, and F1 score to comprehensively evaluate the model's performance; the indicators intuitively reflect the model's performance in different aspects, providing an important reference for subsequent model optimization. S4. Determine the optimal network structure.

2. The truck warning status recognition method based on convolutional neural network according to claim 1, characterized in that: S11 includes the following steps: acquiring GPS data, determining the threshold for the normal operating speed of trucks, and removing abnormal trajectory point data with an average speed greater than the threshold; abnormal latitude and longitude coordinate data that are not within the study area and duplicate data are also screened out. S12 includes the following steps: In order to ensure the consistency and comparability of the data, coordinate transformation is performed using the WGS-84 coordinate system as a unified standard, making the visualization of trajectory data on the map more convenient and intuitive, and facilitating effective analysis and application.

3. The truck warning status recognition method based on convolutional neural networks according to claim 1, characterized in that: S21 includes: The system calculates the truck's operating speed, acceleration, and steering angle and rate of change of steering angle; it detects dangerous behaviors such as sudden braking or acceleration by analyzing instantaneous speed and acceleration changes; it determines whether the truck is turning by analyzing the steering angle and rate of change of steering angle; and it characterizes and analyzes dangerous behaviors such as speeding, fatigued driving, rollover, or collision based on the calculated data. The specific calculations are as follows: The Haversine formula is used to calculate the geographical distance between two adjacent points: a=sine 2 (Δlat / 2)+cos(lat i )*bas(lat i+1 )*sine 2 (Δlon / 2) (1) c = 2 * asin(sqrt(a)) (2) d = R * c (3) Where d represents the spherical distance between the two points, R represents the average radius of the Earth, and lat i and lat i+1 Δlat represents the latitude of two adjacent points, Δlat = lat i+1 -lat i Representing the difference in latitude, Δlon = lon i+1 -lon i Represents the difference in longitude; Acceleration and jerk calculation: a i =Δv / Δt (4) j i =Δa / Δt (5) Calculation of change in direction angle and rate of change: b.c. i =Δθ=θ i+1 -θ i (6) ba i =Δθ / Δt=θ i+1 -θ i / Δt (7) Where Δv represents the change in velocity between adjacent points i and i+1, Δa represents the change in acceleration between adjacent points i and i+1, and Δt represents the change in time.

4. The truck warning status recognition method based on convolutional neural network according to claim 1, characterized in that: In step S23, the trajectory points with an average velocity of 0 are processed separately. The specific steps for this division are as follows: S231. Divide the trajectory into segments by performing short-stroke segmentation; S232. Calculate the basic characteristics of each trajectory point and filter out the trajectory data with a length not less than a certain value that are in one of the four warning states. S233. Sort and number the trajectory points according to the license plate number and time, and make preliminary divisions for different time periods of different vehicles, taking the continuous journey of each vehicle as the short journey division unit. S234. For trajectory points with an average velocity of 0, handle them in different cases. For points with an average velocity of 0 where adjacent trajectory points are all non-zero velocities, set them as breakpoints. For long continuous segments with zero velocity, delete them and divide the trajectory into segments at the next non-zero velocity point, and update the numbering. S235. After the short-stroke trajectory segment is divided, extract the trajectory segment with a length not less than a certain value. S236. Trajectory segment integration and encoding update.

5. The truck warning status recognition method based on convolutional neural network according to claim 1, characterized in that: S32 further includes the following specific construction steps: S321. Transform the trajectories in the same segment into multiple dimensions. The input matrix is ​​in the form of formula (8), where the rows of the matrix represent a single feature, the columns represent all features of the sampling point at the current time, the width of the input matrix is ​​the dimension of the input feature, which is consistent with the dimension of the model input layer, and the length m of the input matrix represents the number of trajectory points contained in a single sample. S322. Make the original trajectory samples have the same length by filling or truncating, and take a trajectory segment with a sliding window M=50; truncate the segments with a length greater than 50, and fill the segments with a length less than 50 with the value of the last trajectory point; S323. An embedding layer fusion technique is introduced to fuse different features of truck operation data; random initialization is used, and the size of the embedding layer is a hyperparameter; the embedding vector of each classification feature is obtained, and they are fused with other types of features through weighted summation, and the fused features are used as input; S324. A two-dimensional convolutional neural network is used, which treats the matrix of multi-dimensional features as image input, thus taking into account the correlation between different feature columns simultaneously. Its convolution formula is as follows: S325. Add residual connections to the network structure. Residual connections connect the input directly to the deep output of the network. S326. Use a batch normalization (BN) layer to standardize the data. Let the input data be: {X1, X2, ..., Xn}. The mean μ of the n input sample data is calculated using formula (10): The variance (sigma) σ of the input sample data is calculated using formula (11). 2 : The input sample data is demeaned and divided by the standard deviation using formula (12): Introducing parameters gama and lambda to scale and shift the standardized data prevents normally distributed standardized data from being distributed on both sides of the sigmoid function and the normalized data from being concentrated in the approximately linear part of the sigmoid function, which would affect the nonlinearity. S327. A pooling layer is periodically inserted between convolutional layers. This invention uses max pooling, selecting the maximum value in each pooling window. The calculation formula is as follows: Among them, z i,k z is the output of the current layer. i-1,k+m This refers to the m-th neuron in the previous layer; S328. The fully connected layer is designed using feature fusion and Dropout techniques. The output of the pooling layer is flattened. The flattening operation stretches the feature map into a one-dimensional vector. The fully connected layer receives the flattened input. Each neuron is connected to each element of the input, and a weighted sum is performed with a bias added. The fully connected layer maps the final output to a linearly separable space and works with the output layer for classification. S329. Finally, the data enters the output layer, which uses a softmax classifier for classification, as shown in the following formula: Ultimately, each position receives a probability value, and the value of the position with the highest probability is used as the recognition result.

6. The truck warning status recognition method based on convolutional neural network according to claim 1, characterized in that: S34 further includes: The specific calculation is as follows: Confusion matrix: The confusion matrix is ​​a two-dimensional matrix that shows the model's predictions on different categories. For multi-class classification problems, the concept of the confusion matrix is ​​expanded. The formula (16) for calculating accuracy is: The formula (17) for calculating precision is: The recall rate is calculated using the formula (18): The formula (19) for calculating the F1 score is as follows: The confusion matrix will contain true positives, false positives, and false negatives for each category.

7. The truck warning status recognition method based on convolutional neural network according to claim 1, characterized in that: S4 includes: The performance of different network configurations is quantitatively evaluated, and key hyperparameter configurations are extracted. Hyperparameters include layer mode, presence or absence of layers, convolutional depth of CNN, and number of filters in layers. By changing the configuration of hyperparameters, the most suitable network structure is finally found. This is then used to identify truck warning status.

Images