Driving intention parameter prediction system and method based on multi-modal data

Abstract

The invention provides a driving intention parameter prediction system and method based on multi-modal data, and the method comprises the steps: designing a weight adjustment mechanism of an LSTM improved attention mechanism model based on the personalized style characteristics of a driver according to different driving style parameters; according to different driving style parameters, attention weights of information corresponding to different historical times are adaptively adjusted, optimization and adjustment of a prediction result are realized, and the accuracy of the prediction result is improved. According to the invention, accurate prediction of the driving behavior can be realized according to human-vehicle-road input information.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of automatic driving of vehicles, and in particular to a driving intention parameter prediction system and method based on multi-modal data. BACKGROUND

[0002] With the development of automatic driving technology, how to make the behavior of automatic driving vehicles more in line with the habits of human drivers has become a research focus. Although existing automatic driving algorithms can simulate human driving behavior to a certain extent, they still have certain limitations in complex and variable traffic environments. In particular, when facing unexpected road conditions or pedestrian behavior, existing automatic driving algorithms often cannot respond reasonably in a timely manner.

[0003] Existing prediction models usually use unified and fixed patterns to learn and assign attention weights from historical data, which cannot dynamically adjust the focus according to the individualized style of the driver, resulting in prediction bias. SUMMARY

[0004] In order to solve the problem of certain limitations of the existing automatic driving algorithm in complex and variable traffic environments in the prior art, the present application provides a driving intention parameter prediction system and method based on multi-modal data.

[0005] The present application is implemented by the following technical solutions: In a first aspect, the present application provides a driving intention parameter prediction system based on multi-modal data, comprising a data acquisition module and an LSTM improved attention mechanism model, wherein the LSTM improved attention mechanism model comprises an encoder and a decoder. The data acquisition module is configured to acquire environmental information, vehicle driving parameters and driver behavior information. The encoder is configured to take the environmental information, vehicle driving parameters and driver behavior information as input sequences to calculate the hidden states of different LSTM units. The decoder is configured to calculate the hidden state of the current time step decoder according to the hidden state of the current time step LSTM unit of the encoder, calculate the attention scores of different LSTM units according to the hidden state of the current time step decoder and the hidden states of different LSTM units in the encoder, and convert them into corresponding first attention weights; adjust the first attention weights according to the driving style parameters to obtain second attention weights of different LSTM units; weight and sum the hidden states of different LSTM units according to the second attention weights to obtain a context vector; and output the context vector as the hidden state of the current time step decoder to obtain a prediction result.

[0006] Preferably, the environmental information includes road conditions, road curvature, and obstacle locations; the vehicle driving parameters include vehicle speed, acceleration, and heading angle; and the driver behavior information includes the driver's eye movement trajectory and head movement data.

[0007] Preferably, the driving style parameter s is obtained by mapping based on the historical statistical data of vehicle driving, and s ∈ [0, 1].

[0008] Furthermore, the historical statistical data of the vehicle's driving is the average acceleration. and maximum steering angle .

[0009] Furthermore, driving styles include conservative, average, and aggressive; conservative: s < 0.3, average: 0.3 ≤ s ≤ 0.7, aggressive: s > 0.7.

[0010] Preferably, the first attention weight is adjusted according to driving style parameters to obtain the second attention weight for different LSTM units, specifically as follows:

[0011] As the first attention weight, As the second attention weight, s For driving style parameters, i For historical time step indexing, n This represents the total length of the historical input sequence.

[0012] Preferably, it also includes a controller that controls the vehicle based on the difference between the predicted result and the set target.

[0013] Furthermore, the controller controls the vehicle based on the difference between the predicted result and the set target. Specifically, the controller assesses the difference between the current vehicle speed and the target vehicle speed based on the throttle and braking force in the predicted result, and fine-tunes the throttle and braking force according to the difference so that the current vehicle speed gradually approaches the target vehicle speed. The controller also adjusts the steering angle based on the steering angle in the predicted result and the current state of the vehicle deviating from the target path.

[0014] Secondly, the present invention provides a method for predicting driving intention parameters based on multimodal data, comprising: The data acquisition module collects environmental information, vehicle driving parameters, and driver behavior information. The environmental information, vehicle driving parameters, and driver behavior information are input as input sequences into the encoder to calculate the hidden states of different LSTM units in the encoder. The hidden state of the decoder at the current time step is calculated based on the hidden state of the LSTM unit at the current time step of the encoder. Based on the hidden state of the decoder at the current time step and the hidden states of different LSTM units in the encoder, attention scores for different LSTM units are calculated and converted into corresponding first attention weights. The first attention weights are adjusted according to driving style parameters to obtain second attention weights for different LSTM units. The hidden states of different LSTM units are weighted and summed according to the second attention weights to obtain a context vector. This context vector is used as the hidden state output of the decoder at the current time step to obtain the prediction result.

[0015] Preferably, the first attention weight is adjusted according to driving style parameters to obtain the second attention weight for different LSTM units, specifically as follows:

[0016] As the first attention weight, As the second attention weight, s For driving style parameters, i For historical time step indexing, n This represents the total length of the historical input sequence.

[0017] Compared with the prior art, the present invention has the following beneficial effects: This invention designs a weight adjustment mechanism for an LSTM-based improved attention mechanism model based on different driving style parameters and the personalized style characteristics of drivers. Different driving styles (such as aggressive, average, or conservative) affect the importance allocation of historical information. To more accurately simulate human driving behavior, this invention's LSTM-based improved attention mechanism model adaptively adjusts the attention weights of information corresponding to different historical times based on different driving style parameters, thereby optimizing the prediction results and improving their accuracy. This invention can accurately predict driving behavior based on input information from the driver, vehicle, and road.

[0018] Furthermore, the controller controls the vehicle based on the difference between the predicted results and the set target, adjusts the vehicle's driving status in real time, and executes assisted driving control to improve the vehicle's driving safety and comfort. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a system structure diagram of the present invention; Figure 2 This is a schematic diagram of the obstacle distribution orientation grid of the present invention; Figure 3 This is a schematic diagram of the encoder-decoder structure based on LSTM of the present invention; Figure 4 This is a flowchart of the LSTM-improved attention mechanism model of the present invention; Figure 5 This is the input-output definition diagram of the LSTM-improved attention mechanism model of this invention; Figure 6 This is a schematic diagram of the encoder-decoder structure adapted to lateral and longitudinal driving styles in this invention. Detailed Implementation

[0021] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention.

[0022] It should be noted that the process equipment or apparatus not specifically mentioned in the following embodiments are all conventional equipment or apparatus in the art.

[0023] It should be noted that the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products, or apparatuses. Furthermore, unless otherwise stated, the numbering of each method step is merely a convenient tool for identifying each method step, and not intended to limit the order of the method steps or define the scope of the invention. Changes or adjustments to their relative relationships, without substantially altering the technical content, should also be considered within the scope of the invention.

[0024] Reference Figure 1 The present invention provides a driving intention parameter prediction system based on multimodal data, comprising: a data acquisition module and an LSTM improved attention mechanism model, wherein the LSTM improved attention mechanism model includes an encoder and a decoder; The data acquisition module is used to collect environmental information, vehicle driving parameters, and driver behavior information; The encoder is used to calculate the hidden state of different LSTM units by taking environmental information, vehicle driving parameters and driver behavior information as input sequences. The decoder is used to calculate the hidden state of the decoder at the current time step based on the hidden state of the LSTM unit at the current time step of the encoder; calculate the attention score of different LSTM units based on the hidden state of the decoder at the current time step and the hidden states of different LSTM units in the encoder, and convert them into corresponding first attention weights; adjust the first attention weights according to driving style parameters to obtain second attention weights for different LSTM units; sum the hidden states of different LSTM units according to the second attention weights to obtain a context vector; and output the hidden state of the decoder at the current time step to obtain the prediction result.

[0025] In some embodiments of the present invention, environmental information includes all information about the driving conditions, such as road curvature, obstacles, road conditions, etc.

[0026] In this invention, road curvature refers to the degree of curvature of the road at a certain moment when a vehicle is traveling on the road surface.

[0027] In this invention, obstacles refer to other vehicles surrounding the vehicle. Using the vehicle as the center and drawing a two-dimensional plane from a top-down perspective, the obstacle area can be divided into eight directions: front, rear, left, right, left-front, left-rear, right-front, and right-rear. Figure 2 As shown, the yellow vehicles are private vehicles, and any vehicle in any of the eight directions divided by the grid is considered an obstacle. In the model of this invention, parameter prediction is affected by environmental factors. If an obstacle is determined to be in a certain direction, it is determined that turning and accelerating in that direction is not possible, ensuring that the driving intention parameters do not conflict with the obstacle.

[0028] In this invention, road conditions refer to whether the road conditions on which the vehicle is traveling are congested (smooth road 0, average speed ≥ 40 km / h; congested road 1, average speed ≤ 20 km / h). The congestion of the road conditions can be determined by navigation software.

[0029] Using precise timestamps to mark each data record allows data from different sources to be compared and analyzed on the same time frame.

[0030] In some embodiments of the present invention, the vehicle driving parameters include the vehicle's speed, acceleration, and heading angle.

[0031] In some embodiments of the present invention, the driver behavior information includes the driver's eye movement trajectory and head movement data.

[0032] In some embodiments of the present invention, the driving style parameter s is obtained based on historical statistical data of vehicle driving, and s ∈ [0, 1]. The historical statistical data of vehicle driving can be the average acceleration. and maximum steering angle .

[0033] Specifically, in this invention, the driving style includes three types: conservative, normal, and aggressive. For the conservative type, the driving style parameter s < 0.3; for the normal type, the driving style parameter s satisfies 0.3 ≤ s ≤ 0.7; and for the aggressive type, the driving style parameter s > 0.7.

[0034] In some embodiments of the present invention, the driving intention parameter prediction system based on multimodal data further includes a controller, which controls the vehicle based on the difference between the prediction result and the set target.

[0035] Specifically, the controller assesses the difference between the current vehicle speed and the target vehicle speed based on the throttle and braking force in the prediction results, and fine-tunes the throttle and braking force according to the difference so that the current vehicle speed gradually approaches the target vehicle speed; the controller adjusts the steering angle based on the steering angle in the prediction results and the current state of the vehicle deviating from the target path.

[0036] The present invention provides a method for predicting driving intention parameters based on multimodal data, comprising: The data acquisition module collects environmental information, vehicle driving parameters, and driver behavior information. The environmental information, vehicle driving parameters, and driver behavior information are input as input sequences into the encoder to calculate the hidden states of different LSTM units in the encoder. The hidden state of the decoder at the current time step is calculated based on the hidden state of the LSTM unit at the current time step of the encoder. Based on the hidden state of the decoder at the current time step and the hidden states of different LSTM units in the encoder, attention scores for different LSTM units are calculated and converted into corresponding first attention weights. The first attention weights are adjusted according to driving style parameters to obtain second attention weights for different LSTM units. The hidden states of different LSTM units are weighted and summed according to the second attention weights to obtain a context vector. This context vector is used as the hidden state output of the decoder at the current time step to obtain the prediction result.

[0037] Specifically, the first attention weight is adjusted based on driving style parameters to obtain the second attention weight for different LSTM units, as follows:

[0038] As the first attention weight, is the second attention weight, s is the driving style parameter, i is the historical time step index, and n is the total length of the historical input sequence.

[0039] In specific implementation, the driving intention parameter prediction method based on multimodal data described in this invention includes the following steps: Step S1: Install multi-channel millimeter-wave radars in the front and rear directions of the vehicle, covering a 360° detection range, with a resolution of not less than 0.1m and a refresh rate of ≥20Hz.

[0040] Environmental information is acquired using millimeter-wave radar installed on the vehicle. This environmental information includes road conditions, road curvature, and obstacle locations.

[0041] The driver's eye movement data and head movement data are collected, and then the visibility coefficient is calculated based on the eye movement data and head movement data. The visibility coefficient is used as the driver's behavior information input to the encoder.

[0042] Using the Smarteye 8.0 eye tracker, the installed camera can recognize and track free head movements within a 360° field of view under various conditions. The output head and left / right eye data is then collected and processed to obtain the driver's eye movement data in different driving scenarios. The basic assumption of eye tracking is that the position, length, and sequence of eye movements are closely related to the attention process; after receiving external stimuli, the trajectory and duration of eye movements can output quantifiable information.

[0043] Eye trackers are used to collect driver eye movement data. The specific method is as follows: The eye tracker is correctly installed and connected to the computer, ensuring the software environment is ready. Then, the eye tracker is calibrated by having the driver fixate on specific calibration points to adjust parameters and ensure data accuracy. Once everything is ready, the data acquisition phase begins, recording the driver's eye movements and monitoring the data stream in real time through the software interface. Recording stops after the test is complete, and the data is exported for subsequent analysis, such as evaluating key indicators like fixation time and frequency, and gaze shift patterns. Throughout the process, the driver must maintain a stable and correct driving posture, and the device may need to be recalibrated if necessary to ensure data accuracy.

[0044] The DBSCAN clustering algorithm was used to identify continuous gaze regions (duration > 200ms), gaze frequency (number of gazes per unit time) was counted, and head motion data (acquired via IMU) was combined to calculate the attention deviation index.

[0045] A visibility coefficient is formed by combining multiple indicators such as fixation frequency, average fixation duration, and gaze deviation.

[0046] in, For eye movement direction, Head direction This is the maximum deflection angle (default 15°). Mean fixation duration (in milliseconds). The preset maximum reasonable gaze time (driving time).

[0047] Step S2: Input the collected environmental information, vehicle driving parameters and driver behavior information into the LSTM (Long Short-Term Memory) improved attention mechanism model and output the prediction results.

[0048] The LSTM improved attention mechanism model includes an encoder and a decoder. The encoder and decoder structures are as follows: Figure 3 As shown.

[0049] In step S201, the encoder takes driver behavior information, vehicle driving parameters and environmental information as input sequences and outputs the hidden states of different LSTM units.

[0050] Here, the encoder step size is set to n, which serves as the input sequence for the LSTM improved attention mechanism model. Figure 5 Specifically, it includes:

[0051] Horizontal information ( ): The vehicle's speed in the Y-axis direction, The rate of change of velocity (acceleration) of the vehicle in the Y-axis direction. The rate of change of vehicle acceleration in the Y-axis direction, Environmental historical information vectors contain environmental information data from a past period. Including steering angle and throttle opening, For the front wheel steering angle, The historical time step represents the time difference between the current moment and a historical moment. These are state transition parameters, used to describe the transition characteristics from one state to another.

[0052]

[0053] Vertical information ( ): The vehicle's velocity along the x-axis, The rate of change of velocity (acceleration) of the vehicle in the x-axis direction. The rate of change of vehicle acceleration in the x-axis direction. Rear wheel slip angle, Front wheel slip angle, The difference between the speed of the vehicle ahead and the speed of your vehicle is used to describe relative speed. Accelerometer measurements reflect the vehicle's longitudinal acceleration. The time constant is used to describe the delay characteristics of the system response. Another time constant is used to describe different system dynamic characteristics.

[0054]

[0055] Environmental information and driver behavior information ( ): Trajectory density, which may be used to describe the distribution of other vehicles on the road. This represents the current road curvature, used to describe the degree of curvature of the road. The coefficient of friction of the road surface affects the vehicle's grip and stability. The visibility coefficient describes the stability of a driver's line of sight during driving. The path tracking coefficient describes the vehicle's ability to follow a preset path. The speed of surrounding vehicles is used to describe the state of surrounding traffic flow.

[0056] Calculate the hidden state based on the input sequence. Specifically:

[0057] in, The encoder is used to input driver behavior information, vehicle driving parameters, and environmental information. This represents the hidden state of an LSTM cell. This represents the hidden state of the LSTM cell of the previous step. For activation function, and These are the input weights for the corresponding LSTM unit. This indicates the bias of the encoder input layer.

[0058] In step S202, the decoder adjusts the attention weights of the hidden states of the LSTM units through the attention mechanism and forms a context vector.

[0059] To prevent the accuracy of the LSTM improved attention mechanism model from decreasing due to the increase in input sequences, a temporal attention mechanism is employed on the decoder to adaptively select the hidden states of relevant LSTM units across all time steps. This mechanism allows the decoder to adaptively select the hidden states of relevant LSTM units across all time steps when generating each output, thereby enhancing the model's ability to understand long sequences. During the decoding phase, whenever a new output word needs to be generated, the decoder not only considers the hidden states of LSTM units from the previous time step but also dynamically focuses on the hidden states of all LSTM units in the encoder through the attention mechanism. For each time step t of the decoder, the similarity between the current hidden state of the decoder and the hidden states of all LSTM units in the encoder needs to be calculated. This is usually called the "attention score," and the formula is as follows:

[0060] In the formula, e tj Let q represent the attention score between the t-th query vector and the j-th key vector. t Let T be the hidden state vector of the decoder, and k be the transpose. j This is the hidden state vector of the LSTM unit in the encoder. Here, the current hidden state of the decoder is usually used as the query and the hidden state of the LSTM unit in the encoder is used as the key.

[0061] After obtaining the attention scores, a Softmax function is typically used to convert them into a probability distribution. This ensures that the sum of all attention scores is 1. The probability distribution represents the importance of the hidden state of each LSTM unit, which is the first attention weight of this invention. In a time series, different historical times have different impacts on the final prediction result, and different driving styles pay different attention to historical information. For example, compared to conservative drivers, aggressive drivers pay less attention to earlier historical information, and this effect is more pronounced the more aggressive the driving style (i.e., the larger the driving style parameter).

[0062] Different driving styles (such as aggressive, normal, or conservative) will affect the allocation of importance of historical information. In order to more accurately simulate human driving behavior, this invention will adaptively adjust the attention weight according to the driving style to optimize the prediction results.

[0063] This invention, after calculating the initial attention weights based on driving style parameters, uses a linear function with the driving style parameters as the slope and intercept to obtain the driving style curve. Since a larger slope of the line indicates a larger change in the independent variable, the difference in the dependent variable is also larger. Utilizing this characteristic, after calculating the initial attention weights, multiplying the initial attention weights of different LSTM units by the corresponding driving style curves reveals the different levels of attention given to different historical data by different styles, thus achieving predictions tailored to specific driving styles. Figure 4 ).

[0064] Calculation of driving style parameter s: based on historical data statistics (average acceleration) Maximum steering angle ), which is mapped to s ∈ [ 0 , 1 ] (conservative type s<0.3, general type 0.3 ≤ s ≤ 0.7, radical type s>0.7).

[0065] Attention weight adjustment:

[0066] Aggressive drivers pay more attention to recent data (weight increases as i approaches n). The first attention weight (the degree of influence of time step i on the prediction result). is the second attention weight, s is the driving style parameter (the larger the value, the more aggressive the driving style), i is the historical time step index (0 = the current time, the larger the i, the further back in time), and n is the total length of the historical input sequence.

[0067] Then, these second attention weights are applied to the hidden states of the corresponding LSTM units and summed in weight to form a context vector that incorporates relevant information from the entire input sequence.

[0068] Step S203: The context vector is used as the hidden state of the decoder at the current time step. During the training phase, to conform to the causal logic of the prediction, the input target sequence is misaligned to train the decoder network.

[0069] During the prediction phase, at each step, the hidden state of the LSTM unit output from the previous step is received to calculate the hidden state of the decoder at the current time step:

[0070] In this formula, This is the output value of the previous long LSTM unit. This represents the hidden state of the inputs to the LSTM units in the first two steps. This represents the hidden state of the output of the previous long LSTM unit. This indicates the hidden state of the output of the LSTM unit at the current step size. The activation function for the output layer of the LSTM unit is the tanh function. The weights of the LSTM unit output layer. This is the bias of the output layer of the LSTM unit. Furthermore, the weight parameters of the input and output layers of the LSTM units in the decoder are set the same as those in the encoder.

[0071] The uncertainty of the predicted values ​​is modeled using a Gaussian distribution, and the mean is output. μ and standard deviation σ The confidence interval is calculated using the formula:

[0072] in, k is the confidence coefficient.

[0073] The improved attention mechanism model of the LSTM outputs interval values ​​instead of constant values, including the turning angle interval. low , high ], throttle range low , high ] and braking force range [ low , high ].

[0074] The width of the prediction interval is dynamically adjusted based on driving style parameters; the prediction interval is narrower for conservative drivers and wider for aggressive drivers.

[0075] The LSTM-based improved attention mechanism model proposed in this invention is a prediction model adapted to both lateral and longitudinal driving styles. Figure 6 A schematic diagram of the encoder-decoder network structure adapted to lateral and longitudinal driving styles.

[0076] In step S3, the controller controls the vehicle based on the difference between the predicted result and the set target, adjusts the vehicle's driving status in real time, and performs assisted driving control, including path planning and driving trajectory adjustment, to improve the vehicle's driving safety and comfort.

[0077] Step S301: The prediction result output from the previous step S2 is converted into the input data for this step and passed to the longitudinally-laterally coupled controller.

[0078] Step S302: Set the corresponding performance index thresholds. When driving on curves, considering safety and stability, the controller will dynamically adjust the target vehicle speed according to the road curvature.

[0079]

[0080] Maximum speed set to To avoid the risk of sideslip caused by excessive lateral acceleration.

[0081] Step S303: Entering the longitudinal control optimization stage, based on the initial predicted throttle and braking forces, the controller first evaluates the difference between the current vehicle speed and the target speed. If the predicted speed is found to be too high or too low, the controller gradually approaches the target speed by fine-tuning the throttle and braking forces. For example, for the third 0.5-second interval, the initial prediction is throttle [20%, 30%] and braking 0%, but to reach the target speed, the controller may decide to reduce the throttle to [15%, 25%] and introduce a slight braking force [2%, 5%] for smooth deceleration. Similarly, for the fourth 0.5-second interval, the throttle is further reduced to [10%, 20%] and the braking force is increased to [7%, 10%] to cope with the sharper curve requirements.

[0082] Step S304, in the lateral control optimization phase, the controller determines the steering angle based on the predicted steering angle. Adjustments are made to the current vehicle status; Assuming the predicted steering angles are 25° and 30°, but the actual path tracking error indicates that the vehicle tends to deviate slightly from the target path, the controller may appropriately increase the steering angle to 28° and 32° respectively, thereby improving the path tracking accuracy.

[0083] Example The information collected by this invention includes: Environmental information: Current road curvature 0.1 (unit: 1 / m), no obstacles in the left front, front, left, and left rear directions, road condition 0; Initial vehicle parameters: speed 60km / h, longitudinal acceleration 0.2m / s², lateral acceleration 0.5m / s²; Driver behavior information: Steering wheel angle is 10°, accelerator force is 35%, and brake force is 0%; These input data were recorded in a continuous time series, once every 0.5 seconds, for a total of 4 seconds of data.

[0084] The first prediction is made using the context vector decoder, outputting the steering angle, throttle force, etc., for the next 0.5 seconds. This step is repeated 3 times, predicting for the next second 0.5 seconds, until all predictions for the next 2 seconds are completed; the prediction result is: Within the first 0.5 seconds, the steering angle is expected to increase to [15°, 20°], the throttle force will remain at [30%, 40%], and the brakes will not be used; In the next 0.5 seconds, the steering angle was further adjusted to [20°, 25°], the throttle force was reduced to [25%, 35%], and the brakes were still not used; Within the next 3.5 seconds, the steering angle is expected to increase to [25°, 30°], the throttle force will remain at [20%, 30%], and the brakes will not be used; In the next 4.5 seconds, the steering angle remains unchanged at [25°, 30°], the throttle force is reduced to [15%, 25%], and the brakes are still not used.

[0085] Performance evaluation of the LSTM-improved attention mechanism model of this invention: The root mean square error (RMSE) is used to quantify the difference between model predictions and actual values. Its calculation formula is as follows:

[0086] The simulation results and visualizations are shown in Table 1.

[0087] Table 1

[0088] Table 1 shows the prediction accuracy of the model proposed in this invention, where a lower RMSE value indicates higher prediction accuracy. The results show that the LSTM-improved attention mechanism model of this invention performs well in conservative and general types and has a certain adaptability to basic scenarios.

[0089] Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A driving intention parameter prediction system based on multimodal data, characterized in that, include: A data acquisition module and an LSTM-improved attention mechanism model, wherein the LSTM-improved attention mechanism model includes an encoder and a decoder; The data acquisition module is used to collect environmental information, vehicle driving parameters, and driver behavior information; The encoder is used to calculate the hidden state of different LSTM units by taking environmental information, vehicle driving parameters and driver behavior information as input sequences. The decoder is used to calculate the hidden state of the decoder at the current time step based on the hidden state of the LSTM unit at the current time step of the encoder, and to calculate the attention score of different LSTM units based on the hidden state of the decoder at the current time step and the hidden state of different LSTM units in the encoder, and convert them into the corresponding first attention weight. The first attention weight is adjusted based on driving style parameters to obtain the second attention weight for different LSTM units; The hidden states of different LSTM units are weighted and summed according to the second attention weights to obtain the context vector; The context vector is used as the hidden state output of the decoder at the current time step to obtain the prediction result.

2. The driving intention parameter prediction system based on multimodal data according to claim 1, characterized in that, The environmental information includes road conditions, road curvature, and obstacle locations; the vehicle driving parameters include vehicle speed, acceleration, and heading angle; and the driver behavior information includes the driver's eye movement trajectory and head movement data.

3. The driving intention parameter prediction system based on multimodal data according to claim 1, characterized in that, The driving style parameter s is obtained by mapping based on the historical statistical data of vehicle driving, and s ∈ [0, 1].

4. The driving intention parameter prediction system based on multimodal data according to claim 3, characterized in that, The historical statistical data of vehicle driving is the average acceleration. and maximum steering angle .

5. The driving intention parameter prediction system based on multimodal data according to claim 3, characterized in that, Driving styles include conservative, average, and aggressive; conservative: s < 0.3, average: 0.3 ≤ s ≤ 0.7, aggressive: s > 0.

7.

6. The driving intention parameter prediction system based on multimodal data according to claim 1, characterized in that, The first attention weight is adjusted based on driving style parameters to obtain the second attention weight for different LSTM units, specifically: As the first attention weight, As the second attention weight, s For driving style parameters, i For historical time step indexing, n This represents the total length of the historical input sequence.

7. The driving intention parameter prediction system based on multimodal data according to claim 1, characterized in that, It also includes a controller that controls the vehicle based on the difference between the predicted result and the set target.

8. The driving intention parameter prediction system based on multimodal data according to claim 7, characterized in that, The controller controls the vehicle based on the difference between the predicted result and the set target. Specifically, the controller evaluates the difference between the current vehicle speed and the target vehicle speed based on the throttle and braking force in the predicted result, and fine-tunes the throttle and braking force according to the difference so that the current vehicle speed gradually approaches the target vehicle speed. The controller also adjusts the steering angle based on the steering angle in the predicted result and the current state of the vehicle deviating from the target path.

9. A method for predicting driving intention parameters based on multimodal data, characterized in that, The prediction system based on claims 1-8 includes: The data acquisition module collects environmental information, vehicle driving parameters, and driver behavior information. The environmental information, vehicle driving parameters, and driver behavior information are input as input sequences into the encoder to calculate the hidden states of different LSTM units in the encoder. The hidden state of the decoder at the current time step is calculated based on the hidden state of the LSTM unit at the current time step of the encoder. Based on the hidden state of the decoder at the current time step and the hidden states of different LSTM units in the encoder, attention scores for different LSTM units are calculated and converted into corresponding first attention weights. The first attention weights are adjusted according to driving style parameters to obtain second attention weights for different LSTM units. The hidden states of different LSTM units are weighted and summed according to the second attention weights to obtain a context vector. This context vector is used as the hidden state output of the decoder at the current time step to obtain the prediction result.

10. The method for predicting driving intention parameters based on multimodal data according to claim 9, characterized in that, The first attention weight is adjusted based on driving style parameters to obtain the second attention weight for different LSTM units, specifically: As the first attention weight, As the second attention weight, s For driving style parameters, i For historical time step indexing, n This represents the total length of the historical input sequence.

Images