Adaptive fog driving speed guidance method based on fusion of multiple factors of cnn-lstm

Abstract

The application relates to a CNN-LSTM-based self-adaptive foggy-day driving speed guiding method based on multi-factor fusion, which comprises the following steps: S1, collecting traffic flow operation data under a highway network connection environment; S2, predicting a speed limit value for safe passing of a fog area of a highway, combining highway foggy-day environment data and a current vehicle state, and using a CNN-LSTM deep learning model to predict an optimal speed limit value of the vehicle in real time; S3, formulating early warning information content; S4, using a vehicle-mounted HMI and a voice prompt function to convey early warning information to a driver; S5, determining a position for publishing fog area information; and S6, pushing early warning information to an intelligent network connection vehicle end. The application can effectively cope with complex foggy-day road environments, has strong scene adaptability, provides omnibearing safety support for the driver, and significantly improves the safety and passing efficiency of foggy-day driving.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of traffic safety, more particularly, to a self-adaptive foggy-day driving speed guidance method based on CNN-LSTM and fusing multiple factors. BACKGROUND

[0002] With the development of artificial intelligence, big data and new generation information technology, the vehicle-road cloud integration technology based on intelligent networked vehicles has made breakthroughs, deeply empowering the traffic system to realize intelligent and safe management and control and intelligent transportation services, and promoting the collaborative evolution of traffic system management and control. However, adverse weather conditions still pose a "long tail" problem for the safe operation of intelligent networked vehicles. Among them, foggy day is one of the typical "long tail" scenarios. In this special environment, the perception uncertainty and driving instability increase significantly, the information acquisition difference of vehicles with different automation levels is highlighted, and the safety risk of traffic system operation increases significantly. How to effectively improve the driving safety in foggy day has become a key problem to be solved.

[0003] Currently, the warning method for foggy-day driving mainly adopts a non-real-time broadcast warning information pushing strategy, that is, obtaining foggy-day information through a meteorological platform, and then publishing foggy-day warning and speed limit reminders through roadside facilities or vehicle-mounted equipment. For example, intelligent traffic facilities (such as variable message boards) publish foggy-day driving tips in the form of text and images at specific points; or relying on networked environment, the adverse weather information published by the meteorological platform is pushed to the vehicle-mounted terminal (usually navigation system) to assist the driver in decision-making. However, the traditional roadside facilities such as variable message boards have low update frequency, which leads to lagged information update and makes it difficult to reflect the current road condition changes in time; and the warning system based on vehicle-mounted equipment relies on the accuracy of meteorological data and monitoring system, which may lead to inaccurate information due to data deviation. More importantly, the existing foggy-day warning adopts a fixed speed limit recommendation strategy, which cannot well match the actual vehicle motion state (such as speed control) and road traffic environment state (such as visibility, traffic flow, distance between front and rear vehicles, etc.) under foggy conditions for dynamic adjustment, making it difficult to meet the driving control expectations of the driver, causing the warning information to be difficult to comply with, and thus failing to truly realize the guidance of vehicle motion and the management and control of traffic operation, ultimately affecting the road traffic capacity.

[0004] The existing fog area speed guidance method is usually based on a simple linear model, mainly relying on the linear relationship between visibility and vehicle speed. However, in the actual fog driving environment, the factors affecting the driving speed of the vehicle are extremely complex and changeable, and visibility is only one of the variables. The environmental factors such as the distance of the preceding vehicle, traffic flow and the operation behavior of the driver will jointly affect the optimal driving speed. Therefore, a single linear model based on visibility cannot fully adapt to the complex fog driving environment, and cannot effectively consider these nonlinear and dynamically changing factors, resulting in insufficient precision and adaptability of vehicle speed guidance, and thus affecting the driving safety. In addition, the existing fog area warning generally lacks effective dynamic human-computer interaction feedback mechanism, and most of them mainly rely on fixed warning signals, prompt sounds or visual warnings, mainly relying on one-way information transmission. In this way, in the dynamic and changing driving environment, it is impossible to provide adjustment suggestions in a timely manner according to the actual operation and reaction of the driver. In the complex fog driving situation, the driver needs to obtain real-time safety driving guidance based on the behavior state and external environmental changes. The existing fog warning fails to fully consider this demand, lacks adaptive and real-time feedback mechanism, and thus the driver cannot obtain sufficient safety guidance or timely adjust the driving behavior, increasing the risk of accidents. SUMMARY

[0005] The technical problem to be solved by the present application is to provide a CNN-LSTM-based multi-factor fusion adaptive fog driving speed guidance method, which can greatly improve the accuracy of vehicle speed prediction and the timeliness of warning, ensure that the warning information can be quickly and accurately conveyed to the driver, improve the user experience, and effectively reduce the incidence of traffic accidents.

[0006] The technical solution adopted by the present application to solve the technical problem is to construct a CNN-LSTM-based multi-factor fusion adaptive fog driving speed guidance method, comprising the following steps:

[0007] S1, collecting traffic flow operation data under the highway network connection environment;

[0008] S2, predicting the speed limit value of safe driving in the fog area of the highway, based on the CNN-LSTM deep learning model, combining the highway fog environment data and the current vehicle state, and predicting the optimal speed limit value of the vehicle in real time;

[0009] S3, formulating warning information content;

[0010] S4, using the vehicle-mounted HMI and voice prompt function to convey the warning information to the driver;

[0011] S5, determining the position of the fog area information release, determining the release node according to the warning information release standard, and ensuring that the driver receives the warning information at the best time and makes a response;

[0012] S6, the intelligent network connected vehicle end is warned, the vehicle speed suggestion, the front fog area distance prompt and the current speed prompt are pushed in real time, and the prompt frequency and intensity are dynamically adjusted according to the feedback of the driver to the guide speed.

[0013] According to the above scheme, in the step S1, the traffic flow operation data includes image data, time series data and traffic density data;

[0014] The image data is obtained by a front vehicle close to the fog area through a vehicle-mounted camera to shoot a fog area image, and the image is transmitted to the background through a network, and the image data includes basic road condition information of a road scene, visibility and lane line;

[0015] The time series data is collected by a GPS module, including a timestamp, vehicle speed, acceleration and distance from the front vehicle;

[0016] The traffic density data is obtained by a traffic management system.

[0017] According to the above scheme, in the step S2, the image data and the time series data in the CNN-LSTM model are fused at the same time scale, and the timestamps of the time series data are expanded by using a linear interpolation method for data with different sampling frequencies;

[0018] For each image data point, the timestamps of the laser radar data and the GPS / OBD-II data are interpolated to the timestamp of the image data by using the linear interpolation formula, and the interpolation step is repeated to ensure that the laser radar and GPS / OBD-II data have corresponding interpolation at each image timestamp.

[0019] According to the above scheme, in the step S2, after the data is processed by using the interpolation method, the data is cleaned and preprocessed before being input into the CNN-LSTM model by using a Kalman filter algorithm.

[0020] According to the above scheme, in the step S2, the image data is normalized.

[0021] According to the above scheme, in the step S3, the warning information content includes a speed limit suggestion, a front fog area distance prompt and a front vehicle distance warning.

[0022] According to the above scheme, the step S4 specifically includes the following steps:

[0023] S401, the optimal driving route is provided for the driver by using the path planning function of the navigation software;

[0024] S402. Through JavaScript secondary development, the prediction results based on the CNN-LSTM model are embedded in the AutoNavi map. A multimodal information release method is adopted, combined with the in-vehicle HMI and voice prompt functions, to provide the driver with speed limit recommendations, fog zone distance prompts, and preceding vehicle distance warnings, thus realizing personalized early warning information push.

[0025] According to the above scheme, in step S5, speed guidance information is issued in advance within a range of 1-2 kilometers before the vehicle enters the fog area, providing the driver with sufficient time to slow down and adjust the driving strategy when the visibility is not yet restricted, and the reminder frequency is adjusted according to the current driver's operation.

[0026] According to the above scheme, in step S5, if the driver does not slow down according to the reminder, the reminder frequency is increased; if the driver slows down according to the reminder, the reminder frequency is reduced;

[0027] At the edge of the fog zone, clear messages are posted to inform drivers of the special driving conditions in the fog zone, and reiterate the need to slow down and maintain a safe distance. Drivers are also clearly reminded that they are about to enter a low-visibility area.

[0028] To ensure drivers receive effective driving guidance, multiple information dissemination points are set up at intervals of 500 to 1 kilometer within foggy areas to continuously push speed and distance warning information to help drivers maintain appropriate driving speeds and safe distances.

[0029] Within 500 meters to 1 kilometer before the exit of the fog area, release points are set up to remind drivers that they are about to exit the fog area and gradually resume normal driving.

[0030] According to the above solution, in step S6, the warning information is conveyed in a multimodal manner through the vehicle-mounted HMI and voice prompts.

[0031] The implementation of the adaptive foggy driving speed guidance method based on CNN-LSTM and integrating multiple factors of the present invention has the following beneficial effects:

[0032] 1. This invention effectively addresses the problem that current fog warnings fail to fully consider the influence of multiple factors. In addition to traditional time-series data such as speed and acceleration, it introduces road traffic environment data (such as the distance to the preceding vehicle and traffic flow) and image data, enabling the model to more comprehensively reflect actual road conditions, thereby improving the accuracy of speed limit recommendations. By combining convolutional neural networks (CNNs) with long-short-term memory networks, the advantages of both in spatial feature extraction and time series modeling are fully utilized. CNN extracts visibility features from image data, while LSTM captures the temporal changes in visibility, speed, acceleration, distance to the preceding vehicle, and traffic flow. It accurately predicts the complex nonlinear interactions between multiple factors, thereby significantly improving the optimal speed prediction and warning capabilities for foggy driving.

[0033] 2. The speed guidance method of the present invention implements adaptive regulation. The proposed speed guidance strategy can dynamically adjust the content and frequency of warnings based on real-time driver feedback on warning information (such as failure to slow down in time or to slow down to the guidance speed) and changes in environmental conditions (such as changes in visibility and vehicle spacing). This adaptive mechanism ensures that drivers receive timely and accurate safety warnings in various complex situations, avoiding the inefficiency and redundant information caused by fixed-mode warnings.

[0034] 3. This invention fills the gap in most existing navigation systems that lack a dedicated warning function in foggy conditions. Through secondary development with AutoNavi Maps, combined with route planning, foggy speed guidance and navigation services, it provides an innovative beyond-visual-range risk warning solution. It can not only provide speed limits in foggy areas, distances to vehicles in front, and prompts for exiting foggy areas, but also optimize route planning in real time. In the complex environment of foggy days, drivers can receive safety warnings in advance, thereby improving driving safety and enhancing their ability to adapt to complex road conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The present invention will be further described below with reference to the accompanying drawings and embodiments, in which:

[0036] Figure 1 It is the technical roadmap of the present invention;

[0037] Figure 2 This is a flowchart of the speed guidance push for highway foggy weather according to the present invention;

[0038] Figure 3 This is a schematic diagram of the CNN-LSTM model structure of the present invention;

[0039] Figure 4 This is a schematic diagram of the early warning interface using Amap in the present invention. DETAILED DESCRIPTION

[0040] In order to have a clearer understanding of the technical features, objectives and effects of the present application, the specific embodiments of the present application will now be described in detail with reference to the drawings.

[0041] As shown in Figures 1-4 The CNN-LSTM-based fusion multi-factor adaptive foggy driving speed guidance method of the present application includes the following steps:

[0042] S1, collect traffic flow operation data under the highway network connection environment. Image data is collected by a vehicle-mounted camera, and the content includes basic road condition information such as road scene, visibility, lane line, etc. In order to ensure that the vehicle accurately captures the current visibility, the front vehicle and the road condition during driving, the image data is captured by the front vehicle near the fog area through the vehicle-mounted camera, and the image is transmitted to the background through the network. The sampling frequency is set to 10-30Hz. The current vehicle-to-vehicle distance is obtained by using a roadside sensing device, and the sampling frequency is set to 1Hz. The time stamp, vehicle speed and acceleration information are collected by a GPS module, and the sampling frequency is set to 5-10Hz. Traffic density data is obtained through a traffic management platform. The update frequency of the warning information is 1-5Hz, which ensures timely response to the operation of the driver and environmental changes.

[0043] The traffic flow operation data collection scheme for the foggy highway scene aims to provide speed dynamic warning information push services for vehicles driving on the highway in foggy weather, to optimize their driving speed and improve road traffic safety.

[0044] S2, predict the speed limit value for safe driving in the fog area of the highway, based on the CNN-LSTM deep learning model, combine the highway foggy environment data and the current vehicle state, and predict the optimal speed limit value of the vehicle in real time, to ensure accurate speed guidance under different road conditions. Based on the CNN-LSTM deep learning model, additional variables under foggy conditions are introduced into the model input variables, including vehicle-to-vehicle distance and traffic density. Compared with the traditional CNN-LSTM model that does not consider these environmental variables, the optimized model can more comprehensively describe the complex feature relationships under foggy conditions, thereby providing more accurate speed guidance under different road conditions and effectively improving the safety and adaptability of foggy driving.

[0045] Since the image data and time series data in the CNN-LSTM model need to be fused at the same time scale, for data with different sampling frequencies, a simple and computationally efficient method of linear interpolation is used to expand the time stamp of the time series data to align with the sampling frequency of the image data, thereby ensuring the consistency of the data in time. The linear interpolation formula is:

[0046]

[0047] In the formula, y1 and y2 are the values corresponding to the known data points, i.e. the front vehicle distance data collected by the lidar or the speed and acceleration information collected by the GPS device, t1 and t2 are the corresponding time stamps, t is the target time point that needs to be interpolated, corresponding to the time stamp of the image data, and y(t) is the value of the target time point after interpolation.

[0048] For each image data point (i.e. each image timestamp), the timestamps of the lidar data and GPS / OBD-II data are interpolated to the timestamp of the image data using the linear interpolation formula described above. The interpolation step is repeated for each image data point (timestamp) to ensure that the lidar and GPS / OBD-II data have corresponding interpolated values at each image timestamp. Finally, all data of different frequencies are aligned according to the timestamps of the image data, ensuring that the data input into the CNN-LSTM model has a consistent time scale.

[0049] After processing the data using the interpolation method, data cleaning and preprocessing is an important step to ensure data quality before inputting the data into the CNN-LSTM model. The main purpose of data cleaning and preprocessing is to remove noise, fill in missing values, standardize data, and ensure data consistency, reliability, and adaptability to model requirements. The Kalman filter algorithm is used in this invention to reduce measurement noise and provide optimal state estimation. The state equation of the Kalman filter describes how the system state at time k (vehicle position, speed, etc.) is transferred to the next time k+1.

[0050] x k =F k x k-1 +B k u k +w k

[0051] In the formula, x k is the system state at time k (vehicle position, speed, etc.); F k is the state transition matrix, representing the process from time k-1 to time k; B k is the control input matrix, used to describe the effect of external control (such as acceleration) on the system; u k is the control input (such as acceleration); w k is the process noise, set as Gaussian white noise.

[0052] The observation equation describes how to obtain the system state from the observation data, and the observation equation is:

[0053] z k =H k x k +v k

[0054] where z k is the observation value (e.g., sensor measurement data) at time k; H k is the observation matrix, representing the relationship between the actual state of the system and the observation value; v k is the observation noise, assumed to be Gaussian white noise. Before starting the filtering, the parameters of the Kalman filter need to be initialized, including the initial state and covariance matrix of the system. The initial state estimate is The initial covariance matrix P0, representing the confidence of the initial state estimate, is usually the identity matrix. The prediction aims to predict the state estimate based on the system state and covariance of the previous step, according to the dynamic model of the system. The formula for the predicted state estimate is:

[0055]

[0056] where is the predicted state estimate. The formula for the predicted covariance matrix is:

[0057]

[0058] where is the predicted covariance matrix, representing the uncertainty of the current state; Q k is the process noise covariance matrix, representing the uncertainty of the model prediction. Then, the update is performed, which combines the actual observation data with the prediction result to obtain a more accurate state estimate. The Kalman gain determines the weighting average proportion of the prediction result and the actual observation result. The calculation formula is:

[0059]

[0060] where K k is the Kalman gain, is the predicted covariance matrix, H k is the observation matrix, P k is the observation noise covariance matrix, representing the uncertainty of the observation data. Based on the Kalman gain, the current state estimate and covariance matrix are updated, with the formula:

[0061]

[0062]

[0063] where is the updated state estimate; z k is the actual observation value; P k is the updated covariance matrix, representing the uncertainty of the current state estimate. Each time there is new observation data z kAt each input, the prediction and update steps are performed to continually refine the state estimate. After each iteration, the state estimate and covariance matrix are updated, providing a more accurate estimate of the system.

[0064] After obtaining the smoothed and denoised data, the data needs to be standardized so that each feature has the same scale. The formula for standardization is:

[0065]

[0066] where Xij is the jth feature value of the ith sample, μj is the mean of the jth feature, and σj is the standard deviation of the jth feature. ij j j

[0067] The collected image data and time series data set is divided into a training set (70%), a validation set (15%), and a test set (15%). Image data needs to be processed through a convolutional neural network (CNN) to extract image features that identify areas with limited visibility. Based on the extracted features, visibility is labeled from low to high as 0-5 categories. Since the size of the image varies, all images need to be adjusted to a uniform format. The present invention sets the image pixel value size to 224x224 and the image format to RGB format (each image has 3 channels: red, green, and blue), i.e., the input image size is 224x224x3. Normalization of the image is performed to scale the pixel value to the range of [0-1], which can improve the efficiency of model training and improve the stability of the model. The formula is:

[0068]

[0069] After image data preprocessing, convolution operation is performed through CNN to extract local features. The convolution layer uses Sobel edge detection to extract spatial features in the image, and the convolution formula is:

[0070]

[0071] where I is the input image, K is the convolution kernel, and (i,j) is the position coordinate of the output image. The pooling layer reduces the image size and computational load, and the maximum pooling formula is:

[0072]

[0073] ​​​where P(i, j) is the pooled image feature; I(i+m, j+n) is the maximum value of the pooling region. After multi-layer convolution and pooling operations, CNN generates a set of feature maps, and the features extracted by the convolutional and pooling layers are combined through a fully connected layer as the output of the CNN. The output layer uses the Softmax activation function to output the final visibility level prediction result. The Softmax activation function converts the score of each class into a probability, and the formula is:

[0074]

[0075] where z i is the score of each class, k is the number of classes, and P(y = c i | x) is the probability that the image belongs to class c i . The time series data features input into the LSTM model include vehicle speed v(t), acceleration a(t), vehicle distance D(t), traffic flow density p(t), and visibility level S(t). The input data at each time step t forms a feature vector. For each time step t, the input of the LSTM is:

[0076] X(t) = [v(t), a(t), D(t), p(t), S(t)]

[0077] In order to enable the model to learn the dependence relationship of a longer time window, a sliding window feature can be constructed. Through the sliding window, a data sequence at each time step can be obtained. The step size of the window is selected to be 1 second, and the size of the sliding window is 5 seconds. The features within each window will be input to the LSTM. For each time step t, the sliding window includes the historical data of the past 5 seconds, i.e.:

[0078]

[0079] where T is the total duration of the data set, W is the size of the sliding window, and each X input (t) is a sequence containing data of the past 5 time steps (5 seconds), with a shape of (5, 5), representing the data features (5 features, such as speed, acceleration, vehicle distance, traffic volume, and visibility level) at each time step. The shape of the LSTM input data is (N, 5, 5), where N is the number of samples generated after the sliding window, i.e. T-W+1, the first 5 is the window size in the time window, and the second 5 is the number of data features at each time step. The core of the LSTM network is to capture the time dependence relationship through the gating mechanism (input gate, forget gate, and output gate), and the formula is as follows:

[0080] Input gate: i t = s(W i · [h t-1 , X t]+b i )

[0081] forget gate: f t =σ(W f ·[h t-1 ,X t ]+b f )

[0082] candidate memory cell:

[0083] memory cell state:

[0084] output gate: o t =σ(W o ·[h t-1 ,X t ]+b o )

[0085] hidden state: h t =o t ·tanh(C t )

[0086] where W f , W i , W C and W o are weight matrices, b i , b f , b C and b o are corresponding bias vectors, tanh is the hyperbolic tangent function, h t-1 is the output of the last time, f t is the retention degree value, C t-1 is the memory state of the last time, i t is the addition degree value of the current time state, is the intermediate state, C t is the current state, o t is the output degree value, h t is the output of the current time, X t is the input of the current time.

[0087] Considering various factors, the model output guides the speed, and the formula is:

[0088]

[0089] where v s(t) is the guiding speed, v(t) is the current speed, a is the adjustment factor of visibility, taking the value of 0.1-0.6, as the visibility level decreases, the speed adjustment factor increases, the value is 0.1 when the visibility is 0 level, and the adjustment factor increases by 0.1 for each increase; β is the adjustment factor of the distance, taking the value of 0.01-0.1, increasing with the decrease of the distance from the front vehicle; γ is the adjustment factor of acceleration, taking the value of 0.2; δ is the adjustment factor of traffic density, taking the value of 0.1-0.3, increasing with the increase of traffic density; D min is the minimum safe distance, taking the distance of 3 seconds at the current speed, m; p max is the maximum traffic density, taking the maximum traffic density of the current scene in the current month, vehicles / km. The guiding speed calculated is the target value of the LSTM model. That is, each input sequence (composed of 5 seconds of historical data) will correspond to a target value h(t), which is the recommended speed that the LSTM model wants to predict at this time step. The recommended speed calculated in the above steps is paired with the input data (i.e. the data of the past 5 seconds) as labels to form a training data set. Each input sequence:

[0090] X input (t) = [X(t-4), X(t-3), X(t-2), X(t-1), X(t)]

[0091] The corresponding target value is:

[0092] h(t) = v l (t), which is the guiding speed output by the LSTM. The output of the LSTM layer is passed to the fully connected layer, and the output of the fully connected layer is the best driving speed recommendation v p (t) at the current time. The fully connected layer linearly transforms the output of the LSTM through a weight matrix and translates through a bias term to obtain the final guiding speed

[0093] v p (t) = W out · h t + b out

[0094] In the formula, W out is the weight matrix of the fully connected layer, with a dimension of 1x1, which needs to be randomly initialized and adjusted through the training process of the model, using the Xavier initialization method; b out is the bias term of the fully connected layer, which is initialized to a small positive number such as 0.1. The loss function is used to measure the error between the predicted speed and the actual speed, and the mean square error (MSE) formula is:

[0095]

[0096] In the formula, vp,i The guided speed v true,i The actual optimal guided speed. MSE measures the average squared difference between the model prediction value and the true value, and the smaller the value, the better the model effect. Next, the model is optimized, that is, the parameters (weights and biases) of the model are adjusted by minimizing the loss function. The Adam (Adaptive Moment Estimation) optimizer is selected for optimization. It is an optimization method based on momentum and adaptive learning rate, which has the ability to automatically adjust the learning rate of each parameter during training. Its formula is:

[0097]

[0098]

[0099] In the formula, β1 and β2 are the decay factors of momentum and adaptive learning rate, η is the learning rate, and ò is a small constant to prevent division by zero. The learning rate is an important hyperparameter that controls the step size of the model parameters at each update, and Adam is used to dynamically adjust the learning rate. After calculating the error of the model through the loss function, the gradient is calculated according to the result of the loss function, and the weights and biases are updated through the backpropagation algorithm. The Adam optimizer updates the weight and bias terms, and the above steps are repeated until the model converges. During training and after training, the validation set is used to evaluate the performance of the model, and the value of MSE is calculated on the validation set to ensure that the model is not overfitting or underfitting. After training is completed, the test set is used to evaluate the final effect of the model and test its generalization ability.

[0100] S3, develop warning information content, including speed limit suggestion, fog area distance ahead warning and vehicle distance warning, information design follows the principles of clarity, explicitness and simplicity, to ensure that drivers can quickly understand and take appropriate action, reducing cognitive burden. The speed limit suggestion is dynamically adjusted according to the current road conditions, visibility in foggy weather and traffic flow state, to ensure that drivers get the most appropriate driving speed prompt. The fog area distance ahead warning and vehicle distance warning information are updated in real time, so that drivers can take appropriate measures in advance. Combined with the feedback of drivers on warning information, the content of warning information is adjusted in real time.

[0101] Taking a distance of 1km from the fog area, a vehicle distance of 20m, and a guided speed of 65km / h as an example, the warning information content is constructed as: "1km ahead into the fog area, vehicle distance 20m, recommended speed 65." Not only should it be concise and clear, but also timely and accurate to deliver to drivers, so as to help them make quick responses in complex foggy environments and ensure driving safety.

[0102] S4, determine the release method of the warning information, through the Gaode mapFigure Two Secondly, the real-time optimal driving route is provided for the driver through the path planning function of Gaode Map, which not only considers the current traffic flow, road conditions, construction information, etc., but also dynamically adjusts the route to help the driver avoid congested sections or accident-prone areas. In the case of low visibility such as fog, the driver is automatically selected to a road with higher visibility and smaller traffic flow to avoid entering an area with higher traffic risk. The real-time positioning function provided by Gaode Map can continuously track the real-time position and direction of the vehicle, provide real-time feedback of the vehicle state, and determine the distance between the vehicle's driving path and the fog area according to the current position of the vehicle. When the vehicle approaches an area with low visibility, relevant warning information will be pushed in advance. On this basis, the prediction results based on the CNN-LSTM model are embedded into Gaode Map through secondary development, thereby realizing personalized warning information push. In order to ensure that the driver can receive warning information comprehensively and timely, a multi-modal information publishing method is adopted, combining the vehicle HMI and voice prompt function to publish speed limit suggestions, fog area distance prompts, and front vehicle distance warnings to the driver, ensuring that the driver can quickly understand and respond.

[0103] Specifically, through the path planning function of Gaode Map Figure Two Secondly, the real-time optimal driving route is provided for the driver through the path planning function of Gaode Map, which not only considers the current traffic flow, road conditions, construction information, etc., but also dynamically adjusts the route to help the driver avoid congested sections or accident-prone areas. In the case of low visibility such as fog, the driver is automatically selected to a road with higher visibility and smaller traffic flow to avoid entering an area with higher traffic risk. The real-time positioning function provided by Gaode Map can continuously track the real-time position and direction of the vehicle, provide real-time feedback of the vehicle state, and determine the distance between the vehicle's driving path and the fog area according to the current position of the vehicle. When the vehicle approaches an area with low visibility, relevant warning information will be pushed in advance. On this basis, the prediction results based on the CNN-LSTM model are embedded into Gaode Map through secondary development, thereby realizing personalized warning information push. In order to ensure that the driver can receive warning information comprehensively and timely, a multi-modal information publishing method is adopted, combining the vehicle HMI and voice prompt function to publish speed limit suggestions, fog area distance prompts, and front vehicle distance warnings to the driver, ensuring that the driver can quickly understand and respond.

[0104] S5, determine the position of the fog area information release, comprehensively consider the current distance from the fog area, vehicle speed and traffic flow situation in front, accurately determine the release node according to the warning information release standard, ensure that the driver can receive the early warning information at the best time and make a response, the driver can receive relevant information outside the visibility range of the fog area, take safety measures in time, realize over-the-horizon warning, so as to significantly improve the driving safety. Within 1-2 kilometers before the vehicle enters the fog area, speed guidance information is released in advance, and the driver can be provided with sufficient time to slow down and adjust the driving strategy when the visibility is not limited. And adjust the prompting frequency according to the current driver's operation, if the driver does not slow down according to the prompt, the prompt frequency is increased; if the driver slows down according to the prompt, the prompt frequency is reduced. At the edge of the entrance of the fog area, an explicit reminder is issued to inform the driver that he is about to enter a low-visibility area. The information released at this place clearly prompts the driver that the fog area has special driving conditions, and emphasizes the importance of slowing down and maintaining a safe distance again. Within the fog area, to ensure that the driver can continuously obtain effective driving guidance, multiple information release points are set at intervals of 500 meters to 1 kilometer, and speed and distance warning information is continuously pushed to help the driver maintain appropriate driving speed and safe distance. Within 500 meters to 1 kilometer before the exit of the fog area, a release point is set to remind the driver that he is about to exit the fog area and gradually restore normal driving state.

[0105] Specifically: the release position of the foggy day warning information of the expressway is set in several key areas. The warning is released 1-2 kilometers before the vehicle enters the fog area to help the driver slow down in advance and prepare. A reminder is issued at the edge of the entrance of the fog area to inform that the low-visibility area will be entered. Multiple release points are set at intervals of 500 meters to 1 kilometer within the fog area to continuously push speed and distance warning information. Within 500 meters to 1 kilometer before the exit of the fog area, a release point is set to remind the driver that he is about to exit the fog area and gradually restore normal driving state.

[0106] S6, the intelligent network connected car end is prewarned, the vehicle-mounted device pushes the speed suggestion, the front fog area distance prompt and the current speed prompt in real time, and dynamically adjusts the prompt frequency and intensity according to the feedback of the driver to the guided speed. For example, when the driver does not slow down in time, the prompt frequency is increased to ensure that the driver receives effective reminders in the driving process. The warning information is conveyed through the multi-modal mode of vehicle-mounted HMI and voice prompt to provide speed guidance for the driver and improve the safety of foggy driving.

[0107] Specifically, early warning information is pushed to the vehicle side, with real-time speed limit recommendations, fog zone distance reminders, and vehicle ahead distance warnings delivered via mobile devices, ensuring drivers receive timely and effective reminders while driving. Early warning information provides comprehensive guidance to drivers through dual feedback methods, including mobile phone display and voice prompts, improving driving safety in foggy conditions. Real-time vehicle driving status and environmental data is obtained through on-board equipment, and combined with the CNN-LSTM deep learning model, speed limit recommendations are dynamically adjusted and optimized. Figure Two This development delivers warning information to the driver in a multi-modal manner of voice and vision through the vehicle's HMI, and adjusts the prompt frequency and intensity in real time based on the driver's feedback on speed guidance, ensuring the timeliness and effectiveness of information transmission, thereby improving driving safety in foggy conditions.

[0108] In a preferred embodiment of the present invention, the visibility information is collected in real time by the onboard camera of the vehicle close to the fog area and uploaded to the backend. The information is combined with the traffic density provided by the traffic management platform, the distance to the preceding vehicle detected by the roadside sensing facilities, and the speed and acceleration data of the vehicle. The CNN-LSTM model is used to output the optimal guidance speed. Figure Two The system was developed for the first time, embedding a trained CNN-LSTM model, integrating path planning and real-time positioning functions, and conveying fog area warning information to drivers in advance in a multi-modal manner of vision and voice through on-board equipment, ensuring that relevant information can be obtained and safety measures can be taken in a timely manner outside the visual range, realizing beyond-visual-range warning, and dynamically adjusting the prompt frequency and intensity based on the driver's feedback on the guidance speed. This not only enhances the driver's ability to cope with complex road conditions in foggy days, but also significantly improves driving safety and convenience.

[0109] In the preferred embodiment of the present application, a deep learning model is constructed by combining a convolutional neural network (CNN) and a long short-term memory network (LSTM). Among them, CNN is mainly used to process image data, which can extract visibility-related features from road images in foggy environment, effectively identify different levels of fog, and provide important image information basis for subsequent speed prediction and warning. LSTM is used to process time series data, which can analyze the time series data of vehicle speed data, vehicle distance change data and traffic flow, capture long-term dependencies and dynamic trends, and better consider the influence of time factors on driving speed. The combination of CNN and LSTM fully utilizes the advantages of the two networks and realizes efficient processing of different types of data. By fusing image and time series data, the interaction of multiple factors is accurately captured, which can greatly improve the accuracy of speed prediction and the timeliness of warning, effectively reducing the incidence of traffic accidents. Considering the speed, acceleration, visibility, vehicle distance and traffic flow, the feature weight is dynamically adjusted, the historical data and real-time feedback are fully utilized, and accurate speed prediction and personalized warning are realized. According to the real-time operation of the driver (such as speeding or ignoring the warning), the change of visibility and the change of vehicle distance, the warning content and frequency are dynamically adjusted, which can adapt to different degrees of foggy environment and various complex road conditions, and provide effective speed guidance for the driver. Based on the high-de Figure Two second development of delivery warning information, with the help of the extensive user base and powerful navigation function of Gaode Map, it ensures that the warning information can be quickly and accurately conveyed to the driver, and improves the user experience. The specific content is as follows:

[0110] 1) CNN-LSTM model-based highway fog speed limit value calculation

[0111] CNN-LSTM model is a deep learning model combining convolutional neural network and long short-term memory network, which can process sequence data containing both spatial features and time dynamic characteristics, and can fully combine the image understanding ability of convolutional neural network and the time modeling ability of long short-term memory network, so that it performs well in processing data with spatial and temporal characteristics, and has significant advantages in complex environment perception applications. Traditional models are usually difficult to handle the interaction of multiple factors, and need to be modeled through linear combination or logical relationship, which is not good at dealing with complex nonlinear relationships and is prone to underfitting problems. The advantage of CNN-LSTM model is that it can learn and represent the complex nonlinear interaction between multiple factors, learn the complex interaction from a large amount of data, and significantly improve the decision quality and accuracy.

[0112] In order to solve the problem of calculating the speed limit value in foggy highway scenes, considering the advantages of CNN-LSTM model in processing image and time series data, the CNN-LSTM model is used to calculate the speed limit value of the vehicle. At the same time, the road traffic environment data is innovatively introduced into the model input variable, including the distance from the front vehicle and the traffic flow, so as to more accurately reflect the complex road condition characteristics. The front vehicle close to the fog area shoots the fog area image through the vehicle-mounted camera, and transmits the image to the background through the network. Based on the collected foggy highway image data, the CNN-LSTM model first takes the image data as input, extracts the features in the image through the CNN layer, and classifies the visibility of the current road based on the visibility grading model. Then, the visibility grading result output by the CNN layer is taken as input, combined with the collected time series data of the vehicle far away from the fog area (including time stamp, vehicle speed, acceleration, distance from the front vehicle and road traffic density), and the data is transmitted to the background through the network and input into the LSTM layer. The LSTM layer models the time dimension of these time series data and visibility grading, captures the dynamic changes and mutual influence of these factors in the time sequence. Finally, the model outputs the best driving speed suggestion under the current environment.

[0113] 2) Based on the high-de Figure Two The early warning information pushing method developed secondly

[0114] In order to solve the adaptability problem of vehicle-mounted equipment in speed guidance, based on the high-de Figure Two The over-the-horizon risk warning method developed secondly. By combining the path planning and real-time positioning function of the high-de, the optimal path planning can be provided for the driver. When the vehicle approaches the fog area, the early warning will be issued, and the distance prompt and speed limit information of the fog area will be provided. When the vehicle enters the fog area, the distance prompt and vehicle speed limit suggestion for driving out of the fog area will be continuously provided, and the prompt frequency and intensity will be dynamically adjusted according to the feedback of the driver to the guided speed. This over-the-horizon warning provides more comprehensive path planning, speed guidance and safety warning for the driver under foggy conditions, effectively improves the response ability of the driver in complex environment, and significantly enhances the driving safety and convenience.

[0115] At present, intelligent networked vehicles are generally equipped with vehicle-mounted display screens, and vehicle-mounted display screens are used as information publishing terminals. By combining display information and voice prompts, timely warning services can be provided for drivers. Not only the path planning and navigation functions are realized, but also the application scenarios are expanded, providing a new solution for the early warning information pushing of foggy highway. At present, the high-de has not provided a special early warning function under foggy conditions, and the application innovatively combines path planning, fog risk prompt and navigation service to provide a safer travel experience for drivers.

[0116] In the preferred embodiment of the present application, the CNN-LSTM model is combined with the Gaode map Figure Two Through secondary development of the Gaode map, the vehicle-mounted device is used as a warning carrier, path planning and real-time positioning are combined, distance prompt, speed limit suggestion and vehicle distance information are provided, the current fog warning is made up, and more intelligent and personalized driving guidance and safety warning are provided for drivers.

[0117] The embodiments of the present application are described above in combination with the drawings, but the present application is not limited to the above specific embodiments, and the above specific embodiments are only illustrative but not restrictive. Those skilled in the art can make many forms under the inspiration of the present application without departing from the purpose of the present application and the scope protected by the claims, and these all belong to the protection of the present application.

Claims

1. A multi-factor adaptive foggy driving speed guidance method based on CNN-LSTM, characterized by: The following steps are involved: S1. Collect traffic flow operation data in the highway network environment; S2: Predicting the speed limit for safe passage in foggy areas on highways. Based on the CNN-LSTM deep learning model, combined with highway foggy environmental data and current vehicle status, the optimal speed limit for vehicles is predicted in real time. The formula for calculating the guide speed is: Where, For the guide speed, is the current speed, The visibility adjustment factor is 0.1-0.

6. As the visibility level increases, the speed adjustment factor increases. When the visibility is level 0, the value is 0.

1. The adjustment factor increases by 0.1 for each level increase. is the adjustment factor of vehicle distance, with a value of 0.01-0.1, and increases as the distance to the preceding vehicle decreases; is the acceleration adjustment factor, which takes a value of 0.2; is the adjustment factor for traffic density, ranging from 0.1 to 0.3, and increases with increasing traffic density; is the minimum safe distance, which is the distance required to travel in 3 seconds at the current speed, m; For the maximum traffic density, take the maximum traffic density of the month in the current scenario, vehicles / km, and the calculated guide speed as the label to be paired with the input data to form the training dataset; The input data includes data from multiple time steps. The time series data features of each time step input into the LSTM model include vehicle speed v(t), acceleration a(t), vehicle distance D(t), traffic flow density ρ(t) and visibility grade S(t); S3. Formulate the content of early warning information; S4. Use the vehicle HMI and voice prompt function to convey warning information to the driver; S5. Determine the location for fog zone information release and determine the release node based on the warning information release standard to ensure that drivers receive the warning information and respond at the best time; S6. Push warning information to intelligent connected vehicles. The on-board equipment will push speed suggestions, distance to the foggy area ahead, and current speed prompts in real time, and dynamically adjust the prompt frequency and intensity based on the driver's feedback on the guidance speed.

2. The adaptive foggy driving speed guidance method based on CNN-LSTM and fusion of multiple factors according to claim 1 is characterized in that: In step S1, the traffic flow operation data includes image data, time series data and traffic density data; The image data is captured by a vehicle-mounted camera in front of the vehicle approaching the foggy area, and the image is transmitted to the backend via the network. The image data includes basic road condition information such as road scene, visibility, and lane lines. The time series data includes the timestamp, vehicle speed, acceleration collected by the GPS module, and the distance between the vehicle and the preceding vehicle collected by the roadside sensing device; The traffic density data is obtained through a traffic management platform.

3. The adaptive foggy driving speed guidance method based on CNN-LSTM and fusion of multiple factors according to claim 2 is characterized in that: In step S2, the image data and time series data in the CNN-LSTM model are fused at the same time scale, and the timestamps of the time series data are expanded using a linear interpolation method for data with different sampling frequencies; For each image data point, the timestamps of the lidar data and GPS / OBD-II data are interpolated to the timestamp of the image data using the linear interpolation formula and the interpolation step is repeated to ensure that the lidar and GPS / OBD-II data have corresponding interpolated values ​​at each image timestamp.

4. The adaptive foggy driving speed guidance method based on CNN-LSTM and fusion of multiple factors according to claim 3 is characterized in that: In step S2, after processing the data using the interpolation method, the data is cleaned and preprocessed using the Kalman filter algorithm before inputting the data into the CNN-LSTM model.

5. The adaptive foggy driving speed guidance method based on CNN-LSTM and integrating multiple factors according to claim 4 is characterized in that: In step S2, the image data is normalized.

6. The adaptive foggy driving speed guidance method based on CNN-LSTM and integrating multiple factors according to claim 1 is characterized in that: In step S3, the warning information includes speed limit suggestions, distance reminders to the foggy area ahead, and distance warnings to the vehicle ahead.

7. The adaptive foggy driving speed guidance method based on CNN-LSTM and fusion of multiple factors according to claim 1 is characterized in that: The step S4 specifically includes the following steps: S401. Providing the driver with a real-time optimal driving route through the route planning function of the navigation software; S402. Through JavaScript secondary development, the prediction results based on the CNN-LSTM model are embedded in the AutoNavi map. A multimodal information release method is adopted, combined with the in-vehicle HMI and voice prompt functions, to provide the driver with speed limit recommendations, fog zone distance prompts, and preceding vehicle distance warnings, thus realizing personalized early warning information push.

8. The adaptive foggy driving speed guidance method based on CNN-LSTM and integrating multiple factors according to claim 1 is characterized in that: In step S5, speed guidance information is issued in advance within a range of 1-2 kilometers before the vehicle enters the fog area, providing the driver with sufficient time to slow down and adjust the driving strategy when the visibility is not yet restricted, and the reminder frequency is adjusted according to the current driver's operation.

9. The adaptive foggy driving speed guidance method based on CNN-LSTM and integrating multiple factors according to claim 8 is characterized in that: In step S5, if the driver does not slow down according to the reminder, the reminder frequency is increased; if the driver slows down according to the reminder, the reminder frequency is reduced; At the edge of the fog zone, clear messages are posted to inform drivers of the special driving conditions in the fog zone, and reiterate the need to slow down and maintain a safe distance. Drivers are also clearly reminded that they are about to enter a low-visibility area. To ensure drivers receive effective driving guidance, multiple information dissemination points are set up at intervals of 500 to 1 kilometer within foggy areas to continuously push speed and distance warning information to help drivers maintain appropriate driving speeds and safe distances. Within 500 meters to 1 kilometer before the exit of the fog area, release points are set up to remind drivers that they are about to exit the fog area and gradually resume normal driving.

10. The adaptive foggy driving speed guidance method based on CNN-LSTM and integrating multiple factors according to claim 1 is characterized in that: In step S6, the warning information is conveyed in a multimodal manner through the vehicle-mounted HMI and voice prompts.

Images