Taxi grading early warning system based on Beidou positioning and multi-dimensional data dynamic fusion

Abstract

The invention relates to the technical field of vehicle safety early warning, in particular to a taxi grading early warning system based on Beidou positioning and multi-dimensional data dynamic fusion, which comprises a Beidou positioning module for capturing the position change of a current taxi in real time; the multivariate data acquisition module is used for acquiring vehicle state data, road environment data, driver behavior data and real-time traffic flow data; the feature extraction and fusion module is used for performing feature extraction on the collected multiple elements, dynamically allocating weights according to the correlation between each feature and the vehicle driving state in different scenes, and performing weighted summation on each feature to obtain the fusion feature of the current taxi; and the safety early warning module is used for analyzing the fusion features and carrying out graded early warning on the safety driving state of the current taxi. The method can rapidly adapt to traffic changes, and can achieve the more comprehensive and more accurate prediction of the safe driving of the taxi.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle safety warning, and more particularly to a taxi grading warning system based on Beidou positioning and multi-dimensional data dynamic fusion. Background Art

[0002] With the increasing busyness of urban traffic, taxis, as an important part of urban public transportation, their driving safety has attracted much attention. In recent years, many technical solutions have been proposed and applied to improve the safety of taxis.

[0003] In the prior art, a relatively common taxi safety warning solution is based on the fusion of single GPS positioning and simple sensor data. This solution obtains the position and speed information of the taxi through an in-vehicle GPS module, and at the same time combines the acceleration sensor and gyroscope sensor in the vehicle to monitor the driving state of the vehicle. For example, when the acceleration sensor detects that the acceleration of the vehicle exceeds a preset threshold, it is determined that the vehicle may be accelerating or braking suddenly; the gyroscope sensor is used to detect the steering angle and angular velocity of the vehicle to determine whether the vehicle has abnormal steering. Once these sensor data exceed the normal range, the system will send corresponding warning information to the driver.

[0004] However, this prior art solution has many drawbacks. First of all, in a complex urban environment, such as areas with high-rise buildings, tunnels or under viaducts, GPS positioning signals are easily blocked or interfered with, resulting in a significant decrease in positioning accuracy or even signal loss. This makes the safety warning system based on GPS positioning unable to accurately determine the position and driving trajectory of the vehicle in these areas, thus affecting the timeliness and accuracy of the warning. For example, in a business district with dense high-rise buildings, due to multiple reflections and attenuations of GPS signals, the system may misjudge the actual driving speed and position of the vehicle. When the vehicle approaches a dangerous area (such as a road construction section), it cannot issue a warning in time, increasing the risk of accidents.

[0005] Secondly, relying solely on the data from simple acceleration sensors and gyroscope sensors for early warning analysis is too one-sided. This solution cannot comprehensively consider the comprehensive impact of road environmental factors, driver behavior characteristics, and real-time traffic flow conditions on the safe driving of taxis. For example, in the case of slippery or icy roads, even if the vehicle's acceleration and steering operations are within the normal sensor thresholds, it may still skid or lose control due to reduced road friction, but the existing system cannot give early warnings based on road environmental information. Similarly, for driver fatigue or distracted driving behaviors (such as using mobile phones for a long time, inattentiveness, etc.), it is difficult to accurately identify and warn only relying on vehicle motion sensors. Moreover, during traffic congestion, when the vehicle starts and stops frequently, simple sensor threshold judgments are prone to generating a large number of false alarms, interfering with the driver's normal driving and reducing the attention paid to early warning information.

[0006] Furthermore, the early warning models in the existing technical solutions are difficult to adjust the parameters and rules of the early warning models in a timely manner according to the traffic characteristics and accident laws of different driving scenarios, cannot quickly adapt to the changes in road traffic, and cannot meet the high-precision requirements for safety early warning of taxis in complex and changeable urban traffic environments.

[0007] Therefore, how to conduct more comprehensive and accurate safety early warning for taxis has become a technical problem that needs to be solved urgently by those skilled in the art. Summary of the Invention

[0008] In view of this, the present invention provides a taxi hierarchical early warning system based on Beidou positioning and multi-dimensional data dynamic fusion, which can quickly adapt to traffic changes and achieve more comprehensive and accurate prediction of the safe driving of taxis.

[0009] To achieve the above object, the present invention adopts the following technical solutions:

[0010] A taxi hierarchical early warning system based on Beidou positioning and multi-dimensional data dynamic fusion, comprising:

[0011] A Beidou positioning module, configured to capture the position change of the current taxi in real time and correct the position information during the satellite signal loss period to obtain Beidou positioning data;

[0012] A multi-source data acquisition module, configured to acquire vehicle status data, road environment data, driver behavior data, and real-time traffic flow data;

[0013] A feature extraction and fusion module, configured to respectively extract features from the Beidou positioning data, vehicle status data, road environment data, driver behavior data, and real-time traffic flow data, and dynamically assign weights according to the correlation between each feature and the vehicle driving state in different scenarios, and perform weighted summation on each feature to obtain the fusion feature of the current taxi;

[0014] A safety warning module, which is used to analyze the fusion features and classify and warn the safe driving state of the current taxi.

[0015] Furthermore, the multi-source data acquisition module includes:

[0016] A vehicle data acquisition unit, which is used to collect the speed, acceleration, steering angle and braking force information of the taxi through on-vehicle sensors;

[0017] A road environment data acquisition unit, which is used to collect the road type, road slope, curve radius of curvature and road sign information of the location where the current taxi is located in real time;

[0018] A driver behavior data acquisition unit, which is used to collect the facial expressions, eye states and head postures of the driver in real time;

[0019] A real-time traffic flow data acquisition unit, which is used to obtain the real-time traffic flow, average vehicle speed and congestion degree of the area where the vehicle is located in real time.

[0020] Furthermore, the feature extraction and fusion module includes:

[0021] A feature extraction unit, which is used to extract features from vehicle state data, road environment data, real-time traffic flow data, driver behavior data and Beidou positioning data to obtain multiple feature vectors;

[0022] A driving scenario analysis unit, which is used to determine the driving scenario at the current moment according to the change conditions of the extracted feature vectors;

[0023] A weight assignment unit, which is used to dynamically adjust the weights of the feature vectors according to the degree of association between the feature vectors and the driving scenario at the current moment;

[0024] A feature fusion unit, which is used to map the feature vectors to the same feature space and perform weighted summation to obtain the fusion feature of the taxi at the current moment.

[0025] Furthermore, when the driving scenario at the current moment changes compared with the previous moment, the formula for the weight assignment unit to adjust the weights of the corresponding feature vectors is:

[0026]

[0027] Wherein, represents the weight of the i-th feature vector at the previous moment t-1; β represents a scalar correction coefficient; represents the weight of the i-th feature vector at the current moment t.

[0028] Furthermore, the calculation method of the scalar correction coefficient β is:

[0029] Construct the road network knowledge graph G road =(V, E), where the node V represents the road element, and each node contains attributes; the edge E represents the road connection relationship; use the graph embedding method to aggregate the node and its neighborhood features into low-dimensional vectors;

[0030] Taking the current position of the taxi as the center, extract the subgraph G within the radius R local ∈G road For the subgraph G road Perform pooling on the nodes in it to generate the local environment feature h local ;

[0031] Input the local environment feature h local into the multi-layer perceptron to obtain the correction coefficient β', so that β' ∈ [0, 1];

[0032] According to the driving scenario at the current moment, positively enhance the scalar correction coefficient of the feature vector with a high degree of relevance to the current driving scenario. At this time, β = β';

[0033] Negatively suppress the scalar correction coefficient of the feature vector with a low degree of relevance to the current driving scenario. At this time, scale the correction coefficient β', and β = 2β' - 1, so that β ∈ [-1, 0].

[0034] Furthermore, the driving scenario at least includes: the normal driving scenario on the urban expressway, the overtaking scenario on the highway, the normal driving scenario on the mountain road, the emergency braking scenario on the urban road, the night driving scenario, the fatigue driving scenario, the driving scenario approaching a dangerous area, the passing scenario in the school area, and the passing scenario in the tunnel.

[0035] Furthermore, the expression of the fusion feature of the taxi at the current moment is:

[0036]

[0037] where F t represents the fusion feature of the taxi at the current moment t, represents the weight of the i-th feature vector at the current moment t, represents the i-th feature vector at the current moment t; n represents a total of n feature vectors.

[0038] Furthermore, the feature extraction unit includes:

[0039] The first feature extraction unit is used to extract features from the vehicle state data, road environment data, real-time traffic flow data, and Beidou positioning data by using a convolutional neural network;

[0040] The second feature extraction unit is used to extract features from the driver behavior data by using a recurrent neural network to obtain the temporal features of the driver behavior.

[0041] Further, the safety warning module makes a classification decision on the fused features through a fully connected layer, outputs a warning level, and issues corresponding warning information according to the warning level; the fully connected layer is provided with four neurons, corresponding to four levels of no warning, mild warning, moderate warning, and severe warning respectively; the warning information is pushed through one or more of an in-vehicle voice system, an in-vehicle display screen, and a mobile phone APP.

[0042] Further, the feature extraction and fusion module and the safety warning module form a deep learning network architecture; when training the deep learning network architecture, a penalty term of Beidou positioning error is introduced into the loss function. When the deviation of the same type of feature vector between the Beidou positioning data and other data sources exceeds a preset value, the loss value is increased, so that the deep learning network architecture pays more attention to the Beidou positioning data. After introducing the penalty term of Beidou positioning error, the loss function is expressed as:

[0043] l = l CE + λl BDS

[0044] where l CE represents the cross-entropy loss function, l BDS represents the penalty term of Beidou positioning error, and λ represents the penalty term weight coefficient;

[0045]

[0046] where N represents the number of batch samples; represents the calculated value of Beidou positioning for a certain feature vector in the i-th sample; represents the calculated value of other data sources for a certain feature vector in the i-th sample; δ represents the difference tolerance threshold of a certain feature vector; |||| represents the two-norm.

[0047] Through the above technical solutions, compared with the prior art, the present invention has the following beneficial effects:

[0048] 1. The present invention combines various types of data such as Beidou positioning data, vehicle state data, road environment data, driver behavior data, and real-time traffic flow data to warn of the driving safety of taxis, deeply explores the data value for different driving scenarios, dynamically adjusts the weights of the data features of each item to highlight the features that need to be focused on and weaken the features that do not need to be focused on, and realizes a more comprehensive and accurate prediction of the safe driving state of taxis by considering multiple dimensions of factors rather than a single indicator.

[0049] 2. The present invention uses Beidou positioning data to replace traditional GPS positioning, showing significant progress in positioning accuracy, anti-interference ability, and global coverage, achieving all-round monitoring and precise early warning of the safe driving of taxis.

[0050] 3. The present invention conducts hierarchical early warning on the safe driving state of taxis to ensure that drivers understand the current safety state, so as to take corresponding measures in a timely manner and reduce the occurrence of safety accidents. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained according to the provided drawings without creative efforts.

[0052] Figure 1 FIG. is the structural framework diagram of the taxi hierarchical early warning system based on Beidou positioning and multi-dimensional data dynamic fusion provided by the present invention;

[0053] Figure 2 FIG. is the detailed architecture diagram of the taxi hierarchical early warning system based on Beidou positioning and multi-dimensional data dynamic fusion provided by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0054] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts belong to the scope of protection of the present invention.

[0055] As Figure 1 - Figure 2 shown, the embodiment of the present invention discloses a taxi hierarchical early warning system based on Beidou positioning and multi-dimensional data dynamic fusion, including: a Beidou positioning module, a multi-source data acquisition module, a feature extraction and fusion module, and a safety early warning module.

[0056] The following will further explain each of the above modules.

[0057] Among them, the Beidou positioning module is used to capture the position change of the current taxi in real time and correct the position information during the satellite signal loss period to obtain Beidou positioning data.

[0058] Specifically, the Beidou positioning module uses a Beidou multi-mode chipset and is installed at a location with good signal reception, such as on the top of a taxi. It simultaneously receives multi-band signals such as B1I and B2a of Beidou to obtain the real-time position information (longitude x, latitude y, altitude h), speed information v, and time information t of the taxi. By using the carrier phase differential technology (RTK) and combining with the data of the ground reference station (obtained in real time through wireless communication), centimeter-level precise positioning is achieved. Its basic principle is to measure the phase difference of the satellite carrier signal between the reference station and the mobile station (taxi), combine with the precise coordinates of the reference station, and calculate the precise position of the taxi using the following formula:

[0059] Wherein, is the phase difference observation value, is the carrier phase received by the mobile station, is the carrier phase received by the reference station, λ is the carrier wavelength, d r is the distance from the mobile station to the satellite, d b is the distance from the reference station to the satellite, ΔN is the integer ambiguity, and ε is the observation noise. By resolving the integer ambiguity ΔN, high-precision relative position information can be obtained. Adding the known coordinates of the reference station, the precise position of the taxi can be obtained. The positioning data update frequency is 10 times per second to ensure that the position changes of the taxi can be captured in real time.

[0060] When the satellite signal is blocked or interfered, it automatically switches to the dead reckoning mode based on the inertial measurement unit (IMU) as a supplement. The IMU includes an accelerometer and a gyroscope. By measuring the acceleration a and angular velocity ω of the vehicle, the position and attitude changes of the vehicle are deduced during the period of satellite signal loss. According to the acceleration a measured by the accelerometer, the displacement increment Δs of the vehicle in a short time is calculated using the formula s = 0.5×a×t 2 (s is the displacement, t is the time interval), and combined with the angle change Δθ = ω×t measured by the gyroscope, the relative position of the vehicle is deduced. Once the satellite signal is restored, it immediately switches back to the Beidou multi-mode positioning mode and calibrates and corrects the dead reckoning result. For example, assuming that n time intervals Δt have passed during the period of satellite signal loss, the total displacement increment Angle change Based on the relative position and attitude obtained by dead reckoning, the initial position information after the restoration of Beidou multi-mode positioning is used for correction to obtain the accurate current position.

[0061] The multi-source data acquisition module is used to collect vehicle status data, road environment data, driver behavior data, and real-time traffic flow data. Specifically, the multi-source data acquisition module includes a vehicle data acquisition unit, a road environment data acquisition unit, a driver behavior data acquisition unit, and a real-time traffic flow data acquisition unit. The following further explains each data acquisition unit.

[0062] A vehicle data acquisition unit for collecting the speed v of a taxi through on-vehicle sensors s , acceleration a s , steering angle θ s and braking force information f b and other information. The speed data is accurate to 0.1 km / h, the acceleration data resolution is 0.1 m / s 2 , the steering angle measurement accuracy reaches 1°, the braking force is represented by the value of the pressure sensor, and the sampling frequency is 10 times per second for all.

[0063] A road environment data acquisition unit for collecting information such as the road type T (such as urban arterial roads, secondary arterial roads, highways, etc., represented in the form of classification codes), road slope (accurate to 0.1°), and curve radius of curvature (accurate to 1 m) of the current location of the taxi in real time by means of a high-precision map and geographic information system (GIS), and using on-vehicle cameras and image recognition technology to detect road sign information S (such as speed limit signs, no overtaking signs, etc.), with an identification accuracy rate of over 95%. Matching these road environment data with the vehicle location information in real time provides a basis for subsequent safety assessment.

[0064] A driver behavior data acquisition unit for installing a driver monitoring system to collect the driver's facial expression E, eye state (such as blink frequency f blink , closed-eye duration t close etc.) and head pose information θ h in real time through a camera; the blink frequency is accurate to the number of times per minute, the closed-eye duration is accurate to 0.1 second, the head pose is represented by the angle change, and the sampling frequency is 5 times per second. Using deep learning algorithms to analyze the driver's facial features to determine whether the driver has behaviors such as fatigue driving and distracted driving (such as using mobile phones, inattentiveness, etc.). For example, when the driver's blink frequency f blink < 10 times / minute and the closed-eye duration t close > 1.5 seconds, combined with features such as unstable head pose, it is determined to be in a suspected state of fatigue driving.

[0065] A real-time traffic flow data acquisition unit, which docks with the data platform of the traffic management department through vehicle networking technology, for obtaining the real-time traffic flow (represented by the number of vehicles passing through per hour), average vehicle speed, and congestion level (represented by a congestion index of 0 - 10, where 0 is smooth and 10 is severely congested) of the area where the vehicle is located in real time. The data update frequency is once per minute. Conducting spatio-temporal correlation analysis on the traffic flow data to extract traffic trend features in the surrounding area of the vehicle for predicting potential dangerous road conditions.

[0066] In a more advantageous embodiment, it further includes a data preprocessing module, which is required to preprocess the collected data. The specific processing methods include:

[0067] Vehicle status data denoising: Denoise the collected vehicle status data, and use the moving average filtering algorithm to remove high-frequency noise interference. Let the vehicle speed data sequence be v s1 , v s2 , ···, v sn , and the effective speed value V st at time t is calculated by the formula:

[0068] where m is the size of the moving window (for example, m = 5). Similarly, data such as acceleration and steering angle are also denoised using a similar moving average filtering algorithm.

[0069] Data standardization and alignment: Standardize data from different sources and of different magnitudes to make them have a unified dimension and data range, facilitating subsequent data fusion and analysis. For example, for speed data, it can be normalized to the interval [0, 1], and the formula is:

[0070] where V n is the normalized speed value, V s is the original speed value, V min and V max are the minimum and maximum values in the speed data respectively. At the same time, all data is aligned according to the timestamp to ensure that different data at the same moment can be accurately corresponding, providing a basis for feature fusion.

[0071] The feature extraction and fusion module is used to extract features from Beidou positioning data, vehicle status data, road environment data, driver behavior data, and real-time traffic flow data respectively, and dynamically assign weights according to the correlation between each feature and the vehicle driving state in different scenarios, and perform weighted summation on each feature to obtain the fusion feature of the current taxi.

[0072] Specifically, the feature extraction and fusion module includes: a feature extraction unit, a driving scenario analysis unit, a weight assignment unit, and a feature fusion unit; the functions of each unit are further described below:

[0073] 1) The feature extraction unit is used to extract features from vehicle status data, road environment data, real-time traffic flow data, driver behavior data, and Beidou positioning data to obtain multiple feature vectors.

[0074] Specifically, the feature extraction unit includes a first feature extraction unit and a second feature extraction unit.

[0075] The first feature extraction unit is used to extract features from vehicle state data, road environment data, real-time traffic flow data, and Beidou positioning data using a convolutional neural network.

[0076] For one-dimensional time series data such as vehicle speed and acceleration, it is first converted into a two-dimensional matrix form (e.g., using a sliding time window method with a window size of 10 sampling points and a step size of 1), and then feature extraction is performed through a one-dimensional convolutional layer. The first convolutional layer is set with 16 convolutional kernels, the kernel size is 3, the step size is 1, and the activation function uses the ReLU function f x = max(0, x), which is used to extract local features. Its calculation formula is:

[0077]

[0078] a j = f(y i )

[0079] Among them, x i is the input data (such as a data point in the speed sequence), k ij is the weight of the jth convolutional kernel, b j is the bias term, y j is the result of the convolution calculation, and a j is the output feature after ReLU activation. Then, data dimensionality reduction is performed through a max pooling layer with a pooling kernel size of 2 and a step size of 2. The calculation formula is: p j = max(a j1 , a j2 )

[0080] Among them, a j1 , a j2 are the two eigenvalue features within the pooling window, and p j is the output after pooling.

[0081] For two-dimensional spatial data such as the curvature radius of a bend and the road slope in the road environment data, a two-dimensional convolutional layer is used for feature extraction. 32 convolutional kernels are set, the kernel size is 3×3, the step size is 1, the activation function is the ReLU function, and then it is reduced in dimension through a max pooling layer (pooling kernel size is 2×2, step size is 2). The two-dimensional convolution calculation formula is:

[0082]

[0083] Among them, x ij is the input two-dimensional data (such as an element in the road slope matrix), k pqij is the weight of the convolutional kernel in the pth row and qth column, b pq is the bias term, y pq is the result of the convolution calculation, and apq is the output feature after ReLU activation.

[0084] Meanwhile, use CNN to extract features from the position information in Beidou multi-mode positioning data, map the longitude and latitude information to a two-dimensional plane, and set specific convolutional kernels to extract position change features, such as the driving trajectory pattern of a vehicle in a specific area. For example, take the longitude and latitude as the horizontal and vertical coordinates of the two-dimensional plane respectively to construct a position matrix, and extract local features in the position matrix through the convolutional layer, such as the feature of the change in the driving trajectory direction of a vehicle within a certain block.

[0085] The second feature extraction unit is used to extract features from the driver behavior data by using a recurrent neural network to obtain the temporal features of the driver behavior.

[0086] Use a recurrent neural network to analyze the driver behavior data. Adopt the long short-term memory network (LSTM) model, set 64 hidden units, and the input is the feature sequence such as the driver's facial expression, eye state, and head posture. The core calculation formulas of the LSTM network include:

[0087] i t = σ(W i · [h t-1 , x t + b i )

[0088] f t = σ(W f · [h t-1 , x t + b f )

[0089] o t = σ(W o · [h t-1 , x t + b o )

[0090]

[0091] h t = o t ⊙ tanh(c t )

[0092] where i t , f t , o t are the activation values of the input gate, forget gate, and output gate respectively, σ is the sigmoid function, W i , W f , W o , W c are the weight matrices, b i, b f , b o , b c is the bias term, h t-1 is the hidden state at the previous moment, x t is the input feature at the current moment (such as the driver's eye state data), ⊙ represents element-wise multiplication, c t is the cell state at the current moment, h t is the hidden state at the current moment. Through the LSTM network, the temporal features of the driver's behavior can be effectively learned, and the change process of behavior patterns such as fatigue driving and distracted driving can be captured. For example, by analyzing the changes in the blink frequency and the duration of eye closure over multiple consecutive time steps, the cumulative effect of the driver's fatigue level can be judged.

[0093] 2) The driving scenario analysis unit is used to determine the driving scenario at the current moment according to the changes in the extracted feature vectors.

[0094] The driving scenarios at least include: normal driving scenarios on urban expressways, overtaking scenarios on highways, normal driving scenarios on mountain roads, emergency braking scenarios on urban roads, night driving scenarios, fatigue driving scenarios, driving scenarios approaching dangerous areas, passing scenarios in school areas, and tunnel passing scenarios.

[0095] The driving scenario analysis unit can use a deep learning model to judge the changes in each feature vector to determine the current driving scenario.

[0096] 3) The weight allocation unit is used to dynamically adjust the weights of each feature vector according to the degree of association between each feature vector and the driving scenario at the current moment.

[0097] There are two ways to adjust the weights. One of them is: using the attention mechanism to fuse multiple features extracted previously. Let e i be the association degree score between the i-th feature (such as traffic flow, vehicle speed, etc.) and the current vehicle driving state. The calculation formula is:

[0098] e i = q T tanh(W e x i + b e )

[0099] Among them, q is the query vector (generated according to the current vehicle driving state), W e is the weight matrix, b e is the bias term, x i is the i-th feature vector.

[0100] Then, the association degree score is normalized through the softmax function to obtain the attention weight a i:

[0101]

[0102] Another way of weight adjustment is as follows: when the driving scenario at the current moment remains unchanged compared with the previous moment, the weights of each feature vector remain unchanged; when the driving scenario at the current moment changes compared with the previous moment, the weight allocation unit needs to adjust the weights of the corresponding feature vectors, and the adjustment formula is:

[0103]

[0104] where, represents the weight of the i-th feature vector at the previous moment t-1; β represents a scalar correction coefficient; represents the weight of the i-th feature vector at the current moment t, and it needs to satisfy

[0105] where, the calculation method of the scalar correction coefficient β is:

[0106] step1. Construct a road network knowledge graph G road =(V, E), where the node V represents a road element, and each node contains attributes; the edge E represents a road connection relationship; use the graph embedding method to aggregate the node and its neighborhood features into a low-dimensional vector.

[0107] step2. Take the current position of the taxi as the center, extract the subgraph G local ∈G road , and perform pooling processing on the nodes in the subgraph G road to generate the local environment feature h local .

[0108] step3. Input the local environment feature h local into a multi-layer perceptron to obtain the correction coefficient β', so that β' ∈ [0, 1]. Among them, the multi-layer perceptron includes an input layer, a hidden layer, and an output layer. Specifically:

[0109] The input of the input layer is the local environment feature h local ;

[0110] The hidden layer contains two layers of fully connected networks, and the activation function is ReLU:

[0111] z1 = ReLU(W1h local + b1)

[0112] z2 = ReLU(W2z1 + b2)

[0113] Output layer: a single-neuron linear layer, and the activation function is Sigmoid:

[0114] β' = σ(W3z2 + b3), and finally, β' ∈ [0, 1].

[0115] Where, W1 represents the weight matrix of the first fully connected layer, b1 represents the bias vector of the first fully connected layer; W2 represents the weight matrix of the second fully connected layer, b2 represents the bias vector of the second fully connected layer; W3 represents the weight matrix of the output layer; b3 represents the bias scalar of the output layer; σ represents the Sigmoid function.

[0116] step4. According to the driving scenario at the current moment, positively enhance the scalar correction coefficient of the feature vector with a high correlation degree with the current driving scenario. At this time, β = β'.

[0117] Negatively suppress the scalar correction coefficient of the feature vector with a low correlation degree with the current driving scenario. At this time, scale the correction coefficient β' so that β' ∈ [0, 0.5], and β = 2β' - 1, so that β ∈ [-1, 0].

[0118] 4) The feature fusion unit is used to map each feature vector to the same feature space and perform weighted summation to obtain the fusion feature of the taxi at the current moment. The expression of the fusion feature of the taxi at the current moment is:

[0119]

[0120] Where, F t represents the fusion feature of the taxi at the current moment t, represents the weight of the i-th feature vector at the current moment t, represents the i-th feature vector at the current moment t; n represents a total of n feature vectors.

[0121] The following explains the weight adjustment method of each feature vector in combination with different scenarios:

[0122] In the scenario of normal driving on an urban expressway, the road environment is relatively stable at this time, the traffic flow is moderate, and the driver's behavior is normal. Beidou positioning and vehicle status data may be dominant, and real-time traffic flow data also needs to be combined to prevent emergencies. At this time, the weights of vehicle status and traffic flow data can be increased, and the weights of other data can be appropriately reduced.

[0123] In the scenario of overtaking on a highway, the vehicle is moving at a high speed, and overtaking behavior may increase risks. At this time, the weights of vehicle status (such as acceleration, steering angle) and driver behavior (such as head posture) need to be increased, and the real-time traffic flow data is also considered to judge the distance between the front and rear vehicles.

[0124] In the normal driving scenario on mountain roads, the roads are winding and the slope changes greatly. The Beidou positioning may be affected by the terrain. It is necessary to increase the weights of road environment data (such as curve curvature, slope) and vehicle status (such as braking force).

[0125] In the emergency braking scenario on urban roads, when an emergency braking occurs suddenly, the weights of vehicle status (acceleration, braking force) and driver behavior (such as facial expression) need to be significantly increased. At the same time, traffic flow data helps to confirm whether the braking is caused by congestion ahead.

[0126] In the night driving scenario, the lighting condition is poor. The weights of driver behavior (such as eye status) and vehicle status (such as headlight use) need to be increased, and the weight of Beidou positioning may be reduced due to the influence of the line of sight.

[0127] In the fatigue driving scenario, long-term driving causes driver fatigue. At this time, the weights of driver behavior (closing eye duration, head posture) are significantly increased, and the vehicle status (such as speed fluctuation) and Beidou positioning (track deviation) may also be adjusted.

[0128] In the emergency braking scenario on highways, the weight of vehicle status characteristics is the highest, which directly reflects the braking intensity. The weight of driver behavior is the second, which helps to confirm whether it is a proactive braking. The weight of road environment data is the lowest because there are generally no special risks on straight roads.

[0129] When the vehicle enters the school area, increase the weight of driver behavior; when the vehicle enters the tunnel, reduce the weight of Beidou positioning and increase the weight of vehicle status.

[0130] The safety warning module is used to analyze the fused features and classify and warn the safe driving status of the current taxi. Specifically, the safety warning module makes classification decisions on the fused features through a fully connected layer, outputs the warning level, and issues corresponding warning information according to the warning level; the fully connected layer is set with four neurons, corresponding to four levels of no warning, mild warning, moderate warning, and severe warning respectively; the softmax function is used as the activation function to convert the output into the probability distribution of each warning level.

[0131] For example, when the probability y4 of the model outputting a severe warning is > 0.8, it is determined to be in a severe warning state, and the corresponding warning mechanism is triggered. During the decision-making process, the location information in the Beidou positioning data can be used to judge whether the vehicle is approaching a dangerous area (such as a high-incidence accident section, a construction area, etc.). If the vehicle is about to enter such an area and other features also indicate the existence of safety risks, the warning level is increased or a warning is issued in advance.

[0132] Generate corresponding warning information according to the safety warning level output by the model. For mild warnings, such as "The vehicle speed is slightly high, please maintain stable driving"; the moderate warning information is "Dangerous driving behavior, please pay attention to decelerating and standardizing operations"; severe warnings issue strong warning information such as "Emergency danger! Immediately take braking or avoidance measures". The warning information is pushed simultaneously through multiple methods such as the in-vehicle voice system, in-vehicle display screen, and mobile phone APP (driver and taxi company management terminal) to ensure that drivers and relevant management personnel can receive it in a timely manner. In the warning information, relevant information about Beidou multi-mode positioning data can be added, such as the prompt of dangerous areas near the current vehicle position (by matching the positioning data with the dangerous area database), to help the driver better understand the surrounding environmental risks.

[0133] In the embodiment of the present invention, the feature extraction and fusion module and the safety warning module form a deep learning network architecture; the process of training and optimizing the deep learning network architecture includes:

[0134] 1) Data division and annotation: Divide the collected multi-source data into a training set, a validation set, and a test set according to a ratio of 7:2:1. Manually annotate the training set data, and determine the corresponding safety warning level for each data sample according to actual accident records and expert experience. For example, samples that have had collision accidents or have serious safety hazards (such as speeding, fatigue driving, and complex road conditions) are marked as severe warnings, and samples with some minor abnormal behaviors (such as occasional speeding within 10%, minor distracted driving) are marked as mild warnings. During the annotation process, fully consider the impact of Beidou multi-mode positioning data on the warning level. For example, even if other data shows normal performance, if the vehicle is in a specific dangerous section (judged by the positioning data), appropriately increase the warning level annotation.

[0135] 2) Loss function and optimization algorithm: Use the cross-entropy loss function to measure the difference between the model prediction result and the true label. The optimization algorithm selects the Adaptive Moment Estimation (Adam) optimization algorithm, sets the initial value of the learning rate to 0.001, and gradually decays during the training process. During the training process, calculate the loss value once for each training batch (batchsize is 32), and update the model parameters through the backpropagation algorithm, adjusting the weights of the convolutional kernel, fully connected layer weights, etc., to gradually reduce the loss value.

[0136] Introduce a penalty term for Beidou positioning error in the loss function. When the deviation of the same type of feature vector between Beidou positioning data and other data sources exceeds the preset value, increase the loss value to make the deep learning network architecture pay more attention to Beidou positioning data. After introducing the penalty term for Beidou positioning error, the loss function is expressed as:

[0137] l = l CE + λl BDS

[0138] Among them, l CE represents the cross-entropy loss function, and l BDS represents the penalty term for the Beidou positioning error, and λ represents the penalty term weight coefficient;

[0139]

[0140] Among them, N represents the number of batch samples; represents the calculated value of the Beidou positioning for a certain feature vector in the i-th sample; represents the calculated value of a certain feature vector in other data sources for the i-th sample; δ represents the difference tolerance threshold of a certain feature vector; |||| represents the two-norm.

[0141] For example, when there is a large deviation between the Beidou positioning data and other data sources (such as when comparing with the vehicle speed data of in-vehicle sensors and the speed difference exceeds a certain threshold), the loss value is increased to prompt the model to pay more attention to the accuracy and consistency of the Beidou positioning data and improve the overall performance of the model.

[0142] 3) Model evaluation and tuning: Regularly evaluate the performance of the model on the validation set, and use indicators such as accuracy, recall rate, and F1 value for measurement. For example, the accuracy rate is the ratio of the number of correctly predicted samples to the total number of predicted samples, the recall rate is the ratio of the number of correctly predicted positive samples to the actual number of positive samples, and the F1 value is the harmonic mean of the accuracy rate and the recall rate.

[0143] According to the evaluation results, adjust and optimize the model architecture and hyperparameters (such as the number of convolutional layer kernels, the number of hidden units, the learning rate, etc.). For example, when it is found that the recall rate of the model for fatigue driving warning is low, increase the number of hidden units of the LSTM network in the behavior analysis layer or adjust the learning rate to improve the model's ability to identify the fatigue behavior of drivers. At the same time, analyze the effect of the Beidou multi-mode positioning data in the model. If it is found that the positioning data makes insufficient contributions to certain warning types (such as vehicle deviation from the normal route warning), adjust the processing method of the positioning data in the feature extraction layer, such as increasing the depth of the convolutional layer or changing the size of the convolutional kernel, to enhance the model's ability to extract the features of the Beidou positioning data.

[0144] In other embodiments, an online optimization module is further included; after the driver receives the warning information, the accuracy of the warning can be feedback through the in-vehicle terminal or the mobile phone APP (such as options like "correct warning", "false alarm", and "unclear warning information"). The online optimization module collects detailed information after an actual accident occurs, including the cause of the accident, vehicle status data at that time, driver behavior data, road environment data, real-time traffic flow data, and Beidou positioning data, and regularly adds this information to the training set to perform online training and optimization on the deep learning network architecture, continuously improving the accuracy and adaptability of the model, enabling it to better handle various complex taxi driving scenarios, and effectively reducing the risk of traffic accidents.

[0145] The present invention uses multiple neural networks to extract features from multivariate data. For example, the convolutional neural network (CNN) is used for algorithms such as the setting of convolutional kernels and pooling strategies for extracting vehicle status and road environment data features, and the recurrent neural network (RNN), especially the long short-term memory network (LSTM), is used for the design of gating units and hidden unit calculation algorithms for analyzing the temporal features of driver behavior data. And through the attention mechanism, an association degree calculation and weight allocation algorithm for dynamically allocating weights of multi-source data to achieve fusion is used to deeply explore the data value, extract complex features and potential patterns, quickly adapt to the current traffic changes, and lay a foundation for accurate warning.

[0146] Through the design and parameter optimization of the deep neural network architecture of the present invention, such as the number of hidden layers, the connection method of neurons, the selection of activation functions, etc., and the design of the loss function, the fine prediction of taxi safe driving is improved.

[0147] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The same or similar parts among the various embodiments can be referred to each other. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method part.

[0148] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be obvious to those skilled in the art. The general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to these embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A taxi classification warning system based on Beidou positioning and dynamic fusion of multi-dimensional data, characterized in that: include: Beidou positioning module is used to capture the current taxi position changes in real time and correct the position information during the period when the satellite signal is lost to obtain Beidou positioning data; Multivariate data collection module, used to collect vehicle status data, road environment data, driver behavior data and real-time traffic flow data; The feature extraction and fusion module is used to extract features from Beidou positioning data, vehicle status data, road environment data, driver behavior data and real-time traffic flow data, and dynamically assign weights based on the correlation between each feature and the vehicle driving status in different scenarios, and perform weighted summation on each feature to obtain the fusion features of the current taxi; The safety warning module is used to analyze the fusion features and issue graded warnings on the current safe driving status of the taxi.

2. The taxi classification warning system based on Beidou positioning and dynamic fusion of multi-dimensional data according to claim 1 is characterized in that: The multivariate data acquisition module comprises: The vehicle data collection unit is used to collect the speed, acceleration, steering angle and braking force information of the taxi through the vehicle-mounted sensors; The road environment data collection unit is used to collect the road type, road slope, curve curvature radius and road sign information of the current taxi location in real time; Driver behavior data collection unit, used to collect the driver's facial expression, eye state and head posture information in real time; The real-time traffic flow data collection unit is used to obtain the real-time traffic flow, average speed and congestion level of the area where the vehicle is located in real time.

3. The taxi classification warning system based on Beidou positioning and dynamic fusion of multi-dimensional data according to claim 1 is characterized in that: The feature extraction and fusion module includes: A feature extraction unit is used to extract features from vehicle status data, road environment data, real-time traffic flow data, driver behavior data and Beidou positioning data to obtain multiple feature vectors; A driving scene analysis unit, used to determine the current driving scene according to the changes of each feature vector extracted; A weight allocation unit, used to dynamically adjust the weight of each feature vector according to the degree of correlation between each feature vector and the current driving scene; The feature fusion unit is used to map each feature vector to the same feature space and perform weighted summation to obtain the fusion feature of the taxi at the current moment.

4. The taxi classification warning system based on Beidou positioning and dynamic fusion of multi-dimensional data according to claim 3 is characterized in that: If the driving scene at the current moment changes relative to the previous moment, the formula for the weight allocation unit to adjust the weight of the corresponding feature vector is: in, represents the weight of the i-th eigenvector at the previous time t-1; β represents a scalar correction coefficient; Represents the weight of the i-th eigenvector at the current time t.

5. The taxi classification warning system based on Beidou positioning and dynamic fusion of multi-dimensional data according to claim 4 is characterized in that: The scalar correction factor β is calculated as: Constructing the road network knowledge graph G road =(V,E), where the node V represents the road element and each node contains attributes; the edge E represents the road connection relationship; the node and its neighborhood features are aggregated into a low-dimensional vector using the graph embedding method; Taking the current location of the taxi as the center, extract the subgraph G within the radius R local ∈G road , for the subgraph G road The nodes in the pool are processed to generate local environment features h local ; The local environment feature h local Input the multilayer perceptron to obtain the correction coefficient β', so that β'∈[0,1]; According to the current driving scene, the scalar correction coefficient of the feature vector with high correlation with the current driving scene is positively enhanced. At this time, β = β'; The scalar correction coefficient of the feature vector with low correlation with the current driving scene is negatively suppressed. At this time, the correction coefficient β' is scaled, and β=2β'-1, so that β∈[-1,0].

6. The taxi classification warning system based on Beidou positioning and dynamic fusion of multi-dimensional data according to claim 3 is characterized in that: The driving scenarios include at least: normal driving scenarios on urban expressways, overtaking scenarios on highways, normal driving scenarios on mountain roads, emergency braking scenarios on urban roads, night driving scenarios, fatigue driving scenarios, driving scenarios approaching dangerous areas, school area traffic scenarios and tunnel traffic scenarios.

7. The taxi classification warning system based on Beidou positioning and dynamic fusion of multi-dimensional data according to claim 3 is characterized in that: The expression of the fusion feature of the taxi at the current moment is: Among them, F t represents the fusion features of the taxi at the current time t, represents the weight of the i-th eigenvector at the current time t, represents the i-th eigenvector at the current time t; n represents a total of n eigenvectors.

8. The taxi classification warning system based on Beidou positioning and dynamic fusion of multi-dimensional data according to claim 3 is characterized in that: The feature extraction unit comprises: A first feature extraction unit is used to extract features from vehicle status data, road environment data, real-time traffic flow data and Beidou positioning data using a convolutional neural network; The second feature extraction unit is used to extract features from the driver behavior data using a recurrent neural network to obtain time series features of the driver behavior.

9. The taxi classification warning system based on Beidou positioning and dynamic fusion of multi-dimensional data according to claim 1 is characterized in that: The safety warning module classifies and decides the fusion features through a fully connected layer, outputs a warning level, and issues corresponding warning information according to the warning level; the fully connected layer is provided with four neurons, corresponding to the four levels of no warning, mild warning, moderate warning and severe warning respectively; The warning information is pushed through one or more of the vehicle voice system, vehicle display screen and mobile phone APP.

10. The taxi classification warning system based on Beidou positioning and dynamic fusion of multi-dimensional data according to claim 1 is characterized in that: The feature extraction and fusion module and the safety warning module form a deep learning network architecture; when the deep learning network architecture is trained, a penalty term of Beidou positioning error is introduced into the loss function. When the deviation of the same type of feature vector between Beidou positioning data and other data sources exceeds a preset value, the loss value is increased, so that the deep learning network architecture pays more attention to Beidou positioning data. After the penalty term of Beidou positioning error is introduced, the loss function is expressed as: l=l CE +l BDS Among them, l CE represents the cross entropy loss function, l BDS represents the penalty term of Beidou positioning error, and λ represents the weight coefficient of the penalty term; Where N represents the number of batch samples; Indicates the calculated value of a certain feature vector in the i-th sample of Beidou positioning; represents the calculated value of a certain feature vector in the i-th sample in other data sources; δ represents the difference tolerance threshold of a certain feature vector; || || represents the binary norm.

Images