Lane changing intention recognition method and system and computer storage medium

Abstract

The invention discloses a lane changing intention recognition method and system and a computer storage medium, and the method comprises the steps: carrying out the data fusion, feature extraction and initial marking processing of collected auxiliary driving information according to a preset time sequence, and carrying out the recognition, so as to execute the lane changing control. The method has the advantages that the problem that an existing marking method lacks robustness is solved, compared with a traditional time window marking method, the recognition time is effectively shortened, the lane changing intention recognition model based on the multi-head attention mechanism is established, and higher accuracy can be obtained; the defects caused by the fact that a time window and other single-feature marking methods need to be overcome in the prior art of a lane changing initial point marking method are overcome, the method is crucial for a driving assistance system to predict lane changing risks and provide timely suggestions, and meanwhile more response time is provided for a driver.

Description

Technical Field

[0001] The present invention relates to the field of information processing technology, and in particular to a lane change intention recognition method, system and computer storage medium. Background Art

[0002] At present, lane change intention recognition considers more the scenario of lanes with same-direction traffic. However, in reality, roads with oncoming traffic account for a larger proportion, and the risk index is higher during the lane changing process. Therefore, research on lane change intention recognition considering oncoming traffic has certain practical value.

[0003] The observation data selection for lane-changing intention recognition is mostly relatively incomplete, and considering the observation factors of the driver's state, such as slight movements of the head and eyes, electromyographic signals of the skin, etc., the robustness of lane-changing intention recognition is low, and eye tracking data, traffic flow data in the same lane, and traffic flow data in the opposite lane are not comprehensively considered. Studies have shown that it has a positive impact on lane-changing intention recognition.

[0004] Moreover, most lane-changing intention recognition models are “black box models” that cannot determine the interactions of various observation data within the model. Instead, an explanatory model is used to analyze in detail the important influencing factors of the lane-changing intention recognition model and the impact of the interactions between the factors.

[0005] In view of the above situation, the present invention proposes a lane change intention recognition method, system and computer storage medium for the lane change scenario of two-way two-lane which is less considered in existing research, which can effectively improve the existing technology and overcome its shortcomings. Summary of the invention

[0006] In view of the shortcomings of the prior art, the present invention provides a lane change intention recognition method, system and computer storage medium to solve the above problems existing in the prior art. The specific solution is as follows:

[0007] In a first aspect, the present invention provides a method for identifying lane change intention, the method comprising:

[0008] Collecting assisted driving information; wherein the assisted driving information includes lane change information and eye movement information;

[0009] fusing the auxiliary driving information according to a preset time sequence, and extracting features from the fused data to form a lane change data set;

[0010] The lane-changing data set is subjected to a start-marking process, and the marked lane-changing data set is identified according to a lane-changing intention recognition model to execute lane-changing control.

[0011] The method of fusing the auxiliary driving information according to a preset time sequence comprises:

[0012] In the preset time series i, the data feature fusion function of the auxiliary driving information is:

[0013]

[0014] fusing the lane change information with the eye movement information;

[0015] Among them, the data feature vector of the lane changing information feature matrix A and the data feature vector of the eye movement information feature matrix B are h(i) and q(i) respectively, con(A,B) is the data feature fusion function of the lane changing information feature matrix A and the eye movement information feature matrix B, L is the total number of data, α and β are initialization parameters.

[0016] The method of performing a start mark process on the lane change data set comprises:

[0017] Track the valley value of the vehicle's lateral displacement position during lane change and determine the corresponding timestamp;

[0018] Search for the highest value of the vehicle moving in the opposite direction of the side shift that is closest to the valley point;

[0019] Search and determine whether there is a behavior of looking at the left rearview mirror before the highest value closest to the valley point, and determine the point closest to the valley point as the starting point.

[0020] The method of tracking the valley value of the lateral displacement position of the vehicle during lane change and determining the corresponding timestamp includes:

[0021] Track and obtain the lane lateral displacement position D of the test vehicle center and the leftmost lane boundary at the time stamp t for the nth lane change n,t The local valley value

[0022] The method of searching for the highest value of the vehicle moving in the opposite direction of the lateral displacement that is closest to the valley point comprises:

[0023] When the test vehicle moves in the opposite direction of the lateral displacement, the highest point closest to the valley value is searched as t prepare1 ∈(t D -δ); where δ is t prepare1 Less than timestamp t D And any real number greater than 0;

[0024] When searching in the opposite direction from the valley value to the lane side, in the interval (t prepare1 ,t ε ) should ensure that 20 consecutive points should be greater than the current point; where t ε For the upward distance t in increasing timestamp prepare1 The timestamp of the 20th most recent point.

[0025] The method of searching and judging whether there is a behavior of looking at the left rearview mirror before the highest value closest to the valley point, and determining the point closest to the valley point as the starting point, comprises:

[0026] Search in t prepare1 Have you ever looked at the left rearview mirror?

[0027] If in t prepare1 Previously, we could search for gaze behavior and mark the closest point as t prepare0 , and mark the starting point as t LC =t prepare0 , the time window of the data set is t LC =t prepare0 ;

[0028] If in t prepare1 No gaze behavior was found before, so the starting point is marked as t LC =t prepare1 , the time window of the data set is t0-t prepare1 ;

[0029] Wherein, the behavior of looking at the left rearview mirror is determined by optical visual detection.

[0030] The method for identifying the marked lane change data set according to the lane change intention recognition model comprises:

[0031] S1. Input lane-changing time series features (x n,t ,y n,t ):

[0032]

[0033] Among them, x n,t is the input feature x at timestamp t n ,y n,t is the input feature label y at timestamp t n ,y n ∈{0:LC,1:LK} is the category label of lane-changing intention, LC is the lane-changing label, and LK is the lane-keeping label;

[0034] S2. Position encoding of input features using position embedding PE x n,t :

[0035] PE (pos,2i) =sin(pos / 10000 2i / d model )

[0036] PE (pos,2i) =cos(pos / 10000 2i / dmodel )

[0037] In the formula, pos represents the position, i represents the dimension, and d model is the dimension of the model;

[0038] For the position offset k between features, PE (pos+k,2i) for:

[0039]

[0040] S3. Multiple different attention head mechanisms are introduced to capture the time series characteristics of lane changes. Each head calculates a set of attention scores, which are merged together to generate the final multi-head attention output; where each head is an independent single-head attention mechanism with different learning parameters;

[0041] S4. Perform global max pooling and regularization on the multi-head attention output;

[0042] The global maximum pooling is as follows: input tensor dimension; for each sample, find the maximum element of each feature channel in the input tensor; obtain a vector of preset length containing the maximum value of each feature channel;

[0043] The regularization simulation is as follows: the dimension of the input tensor; during the training process, for each element in the input tensor, the probability of setting it to 0 is p, and the probability of retaining it is (1-p); generating a binary mask with the same shape as the input tensor, randomly selecting each element; multiplying the elements of the input tensor with the binary mask, setting some elements to 0; obtaining a final filtered output tensor in which some elements are set to 0 and others remain unchanged;

[0044] S5 calculates the likelihood of each type of driving intention, determines the final intention according to the likelihood value of each lane change category, and determines the lane change category with the greatest likelihood as the lane change intention at the current timestamp;

[0045] Among them, the probability of each driving intention It can be calculated by the following formula:

[0046]

[0047] Among them, y n,t is the input feature label y at timestamp t n .

[0048] Each head computes a set of attention scores, which are merged together to generate a final multi-head attention output, the method comprising:

[0049] For each input vector (x n,t ,yn,t ) creates a query vector, a key vector and a value vector; wherein the query vector, the key vector and the value vector are respectively n,t ,y n,t ) is obtained by multiplying the three matrices obtained through training;

[0050] Calculate the intent value for each vector:

[0051]

[0052] Where d k is the dimension of the key vector; Q is the query vector; K is the key vector; T is the lane change duration; v is the value vector;

[0053] Add a multi-head attention mechanism in the self-attention layer, map the query, key, and value matrices through different linear transformations, and concatenate the different matrices together before performing linear transformations:

[0054] MultiHead(Q,K,V)=Concat(head1,…head h )W O

[0055] head i =Attention(QW i Q , KW i K ,VW i V )

[0056] In the formula and h represents the head of attention, and d k =d v =d model / h.

[0057] The method further comprises:

[0058] Generate a perturbed dataset x by perturbing the selected samples n,t , in the perturbed data set x n,t The models are trained on the same network and the differences between the models are measured by the following objective function:

[0059]

[0060] In the formula, f represents the original model; g represents the simple model; G is the set of simple models, π x Represents the data x' in the new dataset n,t With the original data x n,t The distance between them; Ω(g) represents the model complexity g;

[0061] Perturbation x near the sample n,t , the similarity of samples before and after perturbation is obtained according to the cosine similarity of different types of text samples:

[0062]

[0063] Where D(x,z) is the distance between the perturbed instance and the original instance, and σ is the bandwidth that controls the width of the kernel function;

[0064] The cosine similarity is used as the weight to optimize the objective function through linear regression:

[0065]

[0066] Among them, f(z) is the predicted value of the perturbed sample in the original feature, and the predicted value is used as the target value, which is the predicted value in the interpretable space feature; π x (z) is the similarity between the perturbed instance and the original instance; g(z') is the predicted value in the interpretable original feature space.

[0067] In a second aspect, the present invention provides a lane change intention recognition system, the system comprising:

[0068] An acquisition module, used to collect auxiliary driving information; wherein the auxiliary driving information includes lane change information and eye movement information;

[0069] A processing module, used for fusing the auxiliary driving information according to a preset time sequence and extracting features from the fused data to form a lane-changing data set;

[0070] The control module is used to perform a start marking process on the lane change data set and identify the marked lane change data set according to the lane change intention recognition model to perform lane change control.

[0071] In a third aspect, the present invention provides a lane change intention recognition device, the device comprising:

[0072] A communication bus, used to realize the connection and communication between the processor and the memory;

[0073] Memory for storing computer programs;

[0074] A processor is used to execute the computer program to implement the following steps:

[0075] Collecting assisted driving information; wherein the assisted driving information includes lane change information and eye movement information;

[0076] fusing the auxiliary driving information according to a preset time sequence, and extracting features from the fused data to form a lane change data set;

[0077] The lane-changing data set is subjected to a start-marking process, and the marked lane-changing data set is identified according to a lane-changing intention recognition model to execute lane-changing control.

[0078] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method described in the first aspect.

[0079] Beneficial effects: The lane change intention recognition method, system and computer storage medium of the present invention perform identification by performing data fusion, feature extraction and starting marking processing on the collected auxiliary driving information according to a preset time series to execute lane change control. By combining vehicle characteristic parameters and driver eye movement characteristics, the problem of lack of robustness of existing marking methods is overcome. At the same time, compared with the traditional time window marking method (TWL), it effectively reduces the recognition time, establishes a lane change intention recognition model based on a multi-head attention mechanism, and can obtain higher accuracy, overcoming the disadvantages of the existing lane change initial point marking method that needs to overcome the time window and other single feature marking methods. This is crucial for the driving assistance system to predict lane change risks and provide timely suggestions, and also provides more reaction time for the driver. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention, and the embodiments in the drawings do not constitute any limitation to the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0081] Figure 1 It is a flow chart of an embodiment of a method for recognizing lane change intention according to the present invention.

[0082] Figure 2 It is a schematic structural diagram of an embodiment of a lane change intention recognition system of the present invention.

[0083] Figure 3 It is a schematic structural diagram of an embodiment of a lane change intention recognition device of the present invention. DETAILED DESCRIPTION

[0084] The technical solution of the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments, which are preferred embodiments of the present invention. It should be understood that the described embodiments are only part of the embodiments of the present invention, not all of the embodiments; it should be noted that the embodiments of the present invention and the features in the embodiments can be combined with each other without conflict. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0085] The main idea of ​​the technical solution of the embodiment of the present invention is: collecting assisted driving information; fusing the assisted driving information according to a preset time series, and extracting features from the fused data to form a lane change data set; performing a start marking process on the lane change data set, and identifying the marked lane change data set according to a lane change intention recognition model to execute lane change control.

[0086] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0087] Embodiment 1

[0088] An embodiment of the present invention provides a method for recognizing lane change intention. Figure 1 As shown, the lane change intention recognition method may specifically include the following steps:

[0089] Step S101, collecting assisted driving information; wherein the assisted driving information includes lane change information and eye movement information.

[0090] In an embodiment of the present invention, the specific method for collecting auxiliary driving information includes the following steps:

[0091] According to the preset maximum speed of the test vehicle and the maximum speed limit of surrounding vehicles, a two-lane, two-way straight highway environment with no central dividing strip is established.

[0092] More specifically, in view of the fact that existing studies rarely consider the lane-changing scenario of a two-way two-lane road, the present invention application embodiment uses UC-win / Road and Tobii-Pro Lab to establish a two-way two-lane road environment and collect driver eye movement data. The total length of the simulated road is about 3.75 kilometers, including a two-lane two-way straight road without a central divider. At the same time, in accordance with Chinese road regulations, the maximum speed limit of the test vehicle is set to 70km / h, and the maximum speed limit of surrounding vehicles is 60km / h.

[0093] The surrounding vehicles within a preset range of the test vehicle are divided into an initial lane leader vehicle, an initial lane following vehicle, a target lane leader vehicle and a target lane following vehicle, and at a timestamp, only one of the initial lane leader vehicle, the initial lane following vehicle, the target lane leader vehicle and the target lane following vehicle is retained.

[0094] More specifically, the simulator data is preprocessed to divide the surrounding vehicles within 200m of the test vehicle into four types: initial lane leader vehicle (Pre), initial lane follower vehicle (Fol), target lane leader vehicle (O_Pre) and target lane follower vehicle (O_Fol). At a timestamp, there is only one Pre, Fol, O_Fol and O_Pre, and redundant surrounding vehicles are deleted.

[0095] The timestamp, vehicle position coordinates, yaw angle, direction, speed, number of lanes, distance to the left boundary, distance to the right boundary, throttle, brake, user conflict, and steering speed characteristics of the test vehicle are obtained; the speed difference, acceleration difference, and gap distance in the three-dimensional vector direction between the surrounding vehicles and the initial lane vehicles of the test vehicle at each timestamp, as well as the gap distance, acceleration difference, and speed difference in the three-dimensional vector direction between the surrounding vehicles and the target lane vehicles are calculated.

[0096] More specifically, the basic characteristics of the test vehicle SV are obtained, including timestamp, vehicle position coordinates, yaw angle, direction, speed, number of lanes, distance to the left boundary, distance to the right boundary, throttle (existence = 1, non-existence = 0), brake (existence = 1, non-existence = 0), conflict with the user (existence = 1, non-existence = 0), and steering speed; the initial lane interaction information (speed difference, acceleration difference, and gap distance between the SV and the initial lane vehicle at X, Y, and Z) and target lane interaction characteristics (gap distance, acceleration difference, and speed difference between the SV and the target lane vehicle at X, Y, and Z) between the surrounding vehicles and the test vehicle are calculated at each timestamp.

[0097] The eye movement coordinates of the driver's eye movement information are collected, and the visual interest area is divided into a left visual interest area, a middle visual interest area and a right visual interest area according to the eye movement behavior, and the left visual interest area, the middle visual interest area and the right visual interest area are expanded to obtain the driver's eye movement characteristics of the fixation event duration and whether there is annotation behavior in the left visual interest area, the middle visual interest area and the right visual interest area.

[0098] More specifically, in addition to the basic eye movement coordinates, the eye gaze data is processed using Tobii-ProLab processing software and the AOI area of ​​interest is divided into three parts: left, middle and right, namely the left side of AOI, the middle side of AOI and the right side of AOI, based on the eye movement behavior. In order to obtain the gaze points of the left and right rearview mirrors and the front area during the lane change process and reduce the influence of the eye tracking device error, each AOI area is enlarged according to the specific situation of the experiment. The driver's eye movement characteristics are obtained: the duration of the gaze event, whether there is annotation behavior on the left, middle and right sides of the AOI area of ​​interest (presence = 1, absence = 0).

[0099] Step S102: fusing the auxiliary driving information according to a preset time sequence, and extracting features from the fused data to form a lane change data set.

[0100] In an embodiment of the present invention, the method for fusing the auxiliary driving information according to a preset time sequence may specifically include: in a preset time sequence i, according to the data feature fusion function of the auxiliary driving information:

[0101]

[0102] fusing the lane change information with the eye movement information;

[0103] Among them, the data feature vector of the lane changing information feature matrix A and the data feature vector of the eye movement information feature matrix B are h(i) and q(i) respectively, con(A,B) is the data feature fusion function of the lane changing information feature matrix A and the eye movement information feature matrix B, L is the total number of data, α and β are initialization parameters.

[0104] Step S103 , performing a start marking process on the lane-changing data set, and identifying the marked lane-changing data set according to a lane-changing intention recognition model to execute lane-changing control.

[0105] In an embodiment of the present invention, the method for performing start mark processing on the lane change data set may specifically include the following steps:

[0106] First, the valley value of the vehicle's lateral displacement position during lane change is tracked and the corresponding timestamp is determined. Then, the highest value of the vehicle moving in the opposite direction of the lateral displacement closest to the valley point is searched. Finally, it is searched and determined whether there is a behavior of looking at the left rearview mirror before the highest value closest to the valley point, and the closest point is determined as the starting point.

[0107] More specifically, the method of tracking the valley value of the vehicle lateral displacement position during lane changing and determining the corresponding timestamp may include: tracking and obtaining the lane lateral displacement position D of the nth lane change between the center of the test vehicle and the leftmost lane boundary at the timestamp t during lane changing.n,t The local valley value

[0108] More specifically, the method for searching for the highest value of the vehicle moving in the opposite direction of the lateral displacement that is closest to the valley point may specifically include:

[0109] When the test vehicle moves in the opposite direction of the lateral displacement, the highest point closest to the valley value is searched as t prepare1 ∈(t D -δ ) ; where δ is t prepare1 Less than timestamp t D And any real number greater than 0;

[0110] When searching in the opposite direction from the valley value to the lane side, in the interval (t prepare1 ,t ε ) should ensure that 20 consecutive points are greater than the current point; where t ε For the upward distance t in increasing timestamp prepare1 The timestamp of the 20th most recent point.

[0111] More specifically, the method of searching and judging whether there is a behavior of looking at the left rearview mirror before the highest value closest to the valley point, and determining the point closest to the valley point as the starting point may specifically include:

[0112] Search in t prepare1 Have you ever looked at the left rearview mirror?

[0113] If in t prepare1 Previously, we could search for gaze behavior and mark the closest point as t prepare0 , and mark the starting point as t LC =t prepare0 , the time window of the data set is t LC =t prepare0 ;

[0114] If in t prepare1 No gaze behavior was found before, so the starting point is marked as t LC =t prepare1 , the time window of the data set is t0-t prepare1 ;

[0115] Wherein, the behavior of looking at the left rearview mirror is determined by optical visual detection.

[0116] In an embodiment of the present invention, the method for identifying a marked lane change data set according to a lane change intention recognition model may specifically include the following steps:

[0117] S1. Input lane-changing time series features (x n,t ,yn,t ):

[0118]

[0119] Among them, x n,t is the input feature x at timestamp t n ,y n,t is the input feature label y at timestamp t n ,y n ∈{0:LC,1:LK} is the category label of lane-changing intention, LC is the lane-changing label, and LK is the lane-keeping label;

[0120] S2. Position encoding of input features using position embedding PE x n,t :

[0121] PE (pos,2i) =sin(pos / 10000 2i / d model )

[0122] PE (pos,2i) =cos(pos / 10000 2i / d model )

[0123] In the formula, pos represents the position, i represents the dimension, and d model is the dimension of the model;

[0124] For the position offset k between features, PE (pos+k,2i) for:

[0125]

[0126] S3. Multiple different attention head mechanisms are introduced to capture the time series characteristics of lane changes. Each head calculates a set of attention scores, which are merged together to generate the final multi-head attention output; where each head is an independent single-head attention mechanism with different learning parameters;

[0127] S4. Perform global max pooling and regularization on the multi-head attention output;

[0128] The global maximum pooling is as follows: input tensor dimension; for each sample, find the maximum element of each feature channel in the input tensor; obtain a vector of preset length containing the maximum value of each feature channel;

[0129] The regularization simulation is as follows: the dimension of the input tensor; during the training process, for each element in the input tensor, the probability of setting it to 0 is p, and the probability of retaining it is (1-p); generating a binary mask with the same shape as the input tensor, randomly selecting each element; multiplying the elements of the input tensor with the binary mask, setting some elements to 0; obtaining a final filtered output tensor in which some elements are set to 0 and others remain unchanged;

[0130] S5 calculates the likelihood of each type of driving intention, determines the final intention according to the likelihood value of each lane change category, and determines the lane change category with the greatest likelihood as the lane change intention at the current timestamp;

[0131] Among them, the probability of each driving intention It can be calculated by the following formula:

[0132]

[0133] Among them, y n,t is the input feature label y at timestamp t n .

[0134] More specifically, in a specific implementation, each head calculates a set of attention scores, and the method in which the attention scores are merged together to generate the final multi-head attention output may specifically include:

[0135] For each input vector (x n,t ,y n,t ) creates a query vector, a key vector and a value vector; wherein the query vector, the key vector and the value vector are respectively n,t ,y n,t ) is obtained by multiplying the three matrices obtained through training;

[0136] Calculate the intent value for each vector:

[0137]

[0138] Where d k is the dimension of the key vector; Q is the query vector; K is the key vector; T is the lane change duration; v is the value vector;

[0139] Add a multi-head attention mechanism in the self-attention layer, map the query, key, and value matrices through different linear transformations, and concatenate the different matrices together before performing linear transformations:

[0140] MultiHead(Q,K,V)=Concat(head1,…head h )W O

[0141] headi =Attention(QW i Q , KW i K ,VW i V )

[0142] In the formula and h represents the head of attention, and d k =d v =d model / h.

[0143] In an embodiment of the present invention, a perturbed data set x may be generated by perturbing the selected sample. n,t , in the perturbed data set x n,t The models are trained on the same network and the differences between the models are measured by the following objective function:

[0144]

[0145] In the formula, f represents the original model; g represents the simple model; G is the set of simple models, π x Represents the data x' in the new dataset n,t With the original data x n,t The distance between them; Ω(g) represents the model complexity g;

[0146] Perturbation x near the sample n,t , the similarity of samples before and after perturbation is obtained according to the cosine similarity of different types of text samples:

[0147]

[0148] Where D(x,z) is the distance between the perturbed instance and the original instance, and σ is the bandwidth that controls the width of the kernel function;

[0149] The cosine similarity is used as the weight to optimize the objective function through linear regression:

[0150]

[0151] Among them, f(z) is the predicted value of the perturbed sample in the original feature, and the predicted value is used as the target value, which is the predicted value in the interpretable space feature; π x (z) is the similarity between the perturbed instance and the original instance; g(z') is the predicted value in the interpretable original feature space.

[0152] In an optional embodiment of the present invention, in order to evaluate the impact of oncoming traffic on the prediction of the vehicle's lane change intention, the original data set of each driver is divided into two different data sets: a data set containing eye movement features, SV basic features and initial lane interaction features; and a data set containing eye movement features, SV basic features, initial lane interaction features and target lane interaction features. These two different training data sets are used to train different models to evaluate their prediction capabilities.

[0153] In another optional embodiment of the present invention, the prediction results of the test set are compared with the true labels, and then the accuracy, precision, recall and F1 score of the model on the test set are calculated, and on this basis, the effect of the lane change intention recognition model is comprehensively evaluated.

[0154] The calculation formula of accuracy is as follows:

[0155]

[0156] The calculation formula of precision is as follows:

[0157]

[0158] The calculation formula for recall is as follows:

[0159]

[0160] The calculation formula of F1 Score is as follows:

[0161]

[0162] Embodiment 2

[0163] An embodiment of the present invention provides a lane change intention recognition system. Figure 2 As shown, the lane change intention recognition system may specifically include the following modules:

[0164] The acquisition module is used to collect assisted driving information; wherein the assisted driving information includes lane change information and eye movement information.

[0165] In an embodiment of the present invention, the specific method for collecting auxiliary driving information includes the following steps:

[0166] According to the preset maximum speed of the test vehicle and the maximum speed limit of surrounding vehicles, a two-lane, two-way straight highway environment with no central dividing strip is established.

[0167] More specifically, in view of the fact that existing studies rarely consider the lane-changing scenario of a two-way two-lane road, the present invention application embodiment uses UC-win / Road and Tobii-Pro Lab to establish a two-way two-lane road environment and collect driver eye movement data. The total length of the simulated road is about 3.75 kilometers, including a two-lane two-way straight road without a central divider. At the same time, in accordance with Chinese road regulations, the maximum speed limit of the test vehicle is set to 70km / h, and the maximum speed limit of surrounding vehicles is 60km / h.

[0168] The surrounding vehicles within a preset range of the test vehicle are divided into an initial lane leader vehicle, an initial lane following vehicle, a target lane leader vehicle and a target lane following vehicle, and at a timestamp, only one of the initial lane leader vehicle, the initial lane following vehicle, the target lane leader vehicle and the target lane following vehicle is retained.

[0169] More specifically, the simulator data is preprocessed to divide the surrounding vehicles within 200m of the test vehicle into four types: initial lane leader vehicle (Pre), initial lane follower vehicle (Fol), target lane leader vehicle (O_Pre) and target lane follower vehicle (O_Fol). At a timestamp, there is only one Pre, Fol, O_Fol and O_Pre, and redundant surrounding vehicles are deleted.

[0170] The timestamp, vehicle position coordinates, yaw angle, direction, speed, number of lanes, distance to the left boundary, distance to the right boundary, throttle, brake, user conflict, and steering speed characteristics of the test vehicle are obtained; the speed difference, acceleration difference, and gap distance in the three-dimensional vector direction between the surrounding vehicles and the initial lane vehicles of the test vehicle at each timestamp, as well as the gap distance, acceleration difference, and speed difference in the three-dimensional vector direction between the surrounding vehicles and the target lane vehicles are calculated.

[0171] More specifically, the basic characteristics of the test vehicle SV are obtained, including timestamp, vehicle position coordinates, yaw angle, direction, speed, number of lanes, distance to the left boundary, distance to the right boundary, throttle (existence = 1, non-existence = 0), brake (existence = 1, non-existence = 0), conflict with the user (existence = 1, non-existence = 0), and steering speed; the initial lane interaction information (speed difference, acceleration difference, and gap distance between the SV and the initial lane vehicle at X, Y, and Z) and target lane interaction characteristics (gap distance, acceleration difference, and speed difference between the SV and the target lane vehicle at X, Y, and Z) between the surrounding vehicles and the test vehicle are calculated at each timestamp.

[0172] The eye movement coordinates of the driver's eye movement information are collected, and the visual interest area is divided into a left visual interest area, a middle visual interest area and a right visual interest area according to the eye movement behavior, and the left visual interest area, the middle visual interest area and the right visual interest area are expanded to obtain the driver's eye movement characteristics of the fixation event duration and whether there is annotation behavior in the left visual interest area, the middle visual interest area and the right visual interest area.

[0173] More specifically, in addition to the basic eye movement coordinates, the eye gaze data is processed using Tobii-ProLab processing software and the AOI area of ​​interest is divided into three parts: left, middle and right, namely the left side of AOI, the middle side of AOI and the right side of AOI, based on the eye movement behavior. In order to obtain the gaze points of the left and right rearview mirrors and the front area during the lane change process and reduce the influence of the eye tracking device error, each AOI area is enlarged according to the specific situation of the experiment. The driver's eye movement characteristics are obtained: the duration of the gaze event, whether there is annotation behavior on the left, middle and right sides of the AOI area of ​​interest (presence = 1, absence = 0).

[0174] The processing module is used to fuse the auxiliary driving information according to a preset time sequence and extract features from the fused data to form a lane change data set.

[0175] In an embodiment of the present invention, the method for fusing the auxiliary driving information according to a preset time sequence may specifically include: in a preset time sequence i, according to the data feature fusion function of the auxiliary driving information:

[0176]

[0177] fusing the lane change information with the eye movement information;

[0178] Among them, the data feature vector of the lane changing information feature matrix A and the data feature vector of the eye movement information feature matrix B are h(i) and q(i) respectively, con(A,B) is the data feature fusion function of the lane changing information feature matrix A and the eye movement information feature matrix B, L is the total number of data, α and β are initialization parameters.

[0179] The control module is used to perform a start marking process on the lane change data set and identify the marked lane change data set according to the lane change intention recognition model to perform lane change control.

[0180] In an embodiment of the present invention, the method for performing start mark processing on the lane change data set may specifically include the following steps:

[0181] First, the valley value of the vehicle's lateral displacement position during lane change is tracked and the corresponding timestamp is determined. Then, the highest value of the vehicle moving in the opposite direction of the lateral displacement closest to the valley point is searched. Finally, it is searched and determined whether there is a behavior of looking at the left rearview mirror before the highest value closest to the valley point, and the closest point is determined as the starting point.

[0182] More specifically, the method of tracking the valley value of the vehicle lateral displacement position during lane changing and determining the corresponding timestamp may include: tracking and obtaining the lane lateral displacement position D of the nth lane change between the center of the test vehicle and the leftmost lane boundary at the timestamp t during lane changing. n,t The local valley value

[0183] More specifically, the method for searching for the highest value of the vehicle moving in the opposite direction of the lateral displacement that is closest to the valley point may specifically include:

[0184] When the test vehicle moves in the opposite direction of the lateral displacement, the highest point closest to the valley value is searched as t prepare1 ∈(t D -δ); where δ is t prepare1 Less than timestamp t D And any real number greater than 0;

[0185] When searching in the opposite direction from the valley value to the lane side, in the interval (t prepare1 ,t ε ) should ensure that 20 consecutive points are greater than the current point; where t ε For the upward distance t in increasing timestamp prepare1 The timestamp of the 20th most recent point.

[0186] More specifically, the method of searching and judging whether there is a behavior of looking at the left rearview mirror before the highest value closest to the valley point, and determining the point closest to the valley point as the starting point may specifically include:

[0187] Search in t prepare1 Have you ever looked at the left rearview mirror?

[0188] If in t prepare1 Previously, we could search for gaze behavior and mark the closest point as t prepare0 , and mark the starting point as t LC =t prepare0 , the time window of the data set is t LC =t prepare0 ;

[0189] If in t prepare1 No gaze behavior was found before, so the starting point is marked as t LC =t prepare1, the time window of the data set is t0-t prepare1 ;

[0190] Wherein, the behavior of looking at the left rearview mirror is determined by optical visual detection.

[0191] In an embodiment of the present invention, the method for identifying a marked lane change data set according to a lane change intention recognition model may specifically include the following steps:

[0192] S1. Input lane-changing time series features (x n,t ,y n,t ):

[0193]

[0194] Among them, x n,t is the input feature x at timestamp t n ,y n,t is the input feature label y at timestamp t n ,y n ∈{0:LC,1:LK} is the category label of lane-changing intention, LC is the lane-changing label, and LK is the lane-keeping label;

[0195] S2. Position encoding of input features using position embedding PE x n,t :

[0196] PE (pos,2i) =sin(pos / 10000 2i / d model )

[0197] PE (pos,2i) =cos(pos / 10000 2i / d model )

[0198] In the formula, pos represents the position, i represents the dimension, and d model is the dimension of the model;

[0199] For the position offset k between features, PE (pos+k,2i) for:

[0200]

[0201] S3. Multiple different attention head mechanisms are introduced to capture the time series characteristics of lane changes. Each head calculates a set of attention scores, which are merged together to generate the final multi-head attention output; where each head is an independent single-head attention mechanism with different learning parameters;

[0202] S4. Perform global max pooling and regularization on the multi-head attention output;

[0203] The global maximum pooling is as follows: input tensor dimension; for each sample, find the maximum element of each feature channel in the input tensor; obtain a vector of preset length containing the maximum value of each feature channel;

[0204] The regularization simulation is as follows: the dimension of the input tensor; during the training process, for each element in the input tensor, the probability of setting it to 0 is p, and the probability of retaining it is (1-p); generating a binary mask with the same shape as the input tensor, randomly selecting each element; multiplying the elements of the input tensor with the binary mask, setting some elements to 0; obtaining a final filtered output tensor in which some elements are set to 0 and others remain unchanged;

[0205] S5 calculates the likelihood of each type of driving intention, determines the final intention according to the likelihood value of each lane change category, and determines the lane change category with the greatest likelihood as the lane change intention at the current timestamp;

[0206] Among them, the probability of each driving intention It can be calculated by the following formula:

[0207]

[0208] Among them, y n,t is the input feature label y at timestamp t n .

[0209] More specifically, in a specific implementation, each head calculates a set of attention scores, and the method in which the attention scores are merged together to generate the final multi-head attention output may specifically include:

[0210] For each input vector (x n,t ,y n,t ) creates a query vector, a key vector and a value vector; wherein the query vector, the key vector and the value vector are respectively n,t ,y n,t ) is obtained by multiplying the three matrices obtained through training;

[0211] Calculate the intent value for each vector:

[0212]

[0213] Where d k is the dimension of the key vector; Q is the query vector; K is the key vector; T is the lane change duration; v is the value vector;

[0214] Add a multi-head attention mechanism in the self-attention layer, map the query, key, and value matrices through different linear transformations, and concatenate the different matrices together before performing linear transformations:

[0215] MultiHead(Q,K,V)=Concat(head1,…head h )W O

[0216] head i =Attention(QW i Q , KW i K ,VW i V )

[0217] In the formula and h represents the head of attention, and d k =d v =d model / h.

[0218] In an embodiment of the present invention, a perturbed data set x may be generated by perturbing the selected sample. n,t , in the perturbed data set x n,t The models are trained on the same network and the differences between the models are measured by the following objective function:

[0219]

[0220] In the formula, f represents the original model; g represents the simple model; G is the set of simple models, π x Represents the data x' in the new dataset n,t With the original data x n,t The distance between them; Ω(g) represents the model complexity g;

[0221] Perturbation x near the sample n,t , the similarity of samples before and after perturbation is obtained according to the cosine similarity of different types of text samples:

[0222]

[0223] Where D(x,z) is the distance between the perturbed instance and the original instance, and σ is the bandwidth that controls the width of the kernel function;

[0224] The cosine similarity is used as the weight to optimize the objective function through linear regression:

[0225]

[0226] Among them, f(z) is the predicted value of the perturbed sample in the original feature, and the predicted value is used as the target value, which is the predicted value in the interpretable space feature; πx (z) is the similarity between the perturbed instance and the original instance; g(z') is the predicted value in the interpretable original feature space.

[0227] In an optional embodiment of the present invention, in order to evaluate the impact of oncoming traffic on the prediction of the vehicle's lane change intention, the original data set of each driver is divided into two different data sets: a data set containing eye movement features, SV basic features and initial lane interaction features; and a data set containing eye movement features, SV basic features, initial lane interaction features and target lane interaction features. These two different training data sets are used to train different models to evaluate their prediction capabilities.

[0228] In another optional embodiment of the present invention, the prediction results of the test set are compared with the true labels, and then the accuracy, precision, recall and F1 score of the model on the test set are calculated, and on this basis, the effect of the lane change intention recognition model is comprehensively evaluated.

[0229] The calculation formula of accuracy is as follows:

[0230]

[0231] The calculation formula of precision is as follows:

[0232]

[0233] The calculation formula for recall is as follows:

[0234]

[0235] The calculation formula of F1 Score is as follows:

[0236]

[0237] Embodiment 3

[0238] An embodiment of the present invention provides a lane change intention recognition device, such as Figure 3 As shown, the lane change intention recognition device may specifically include the following modules:

[0239] A communication bus, used to realize the connection and communication between the processor and the memory;

[0240] The memory is used to store computer programs; the memory may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk memory. The memory may optionally include at least one storage device.

[0241] A processor is used to execute the above computer program to implement the following steps:

[0242] First, assisted driving information is collected; wherein the assisted driving information includes lane change information and eye movement information.

[0243] In an embodiment of the present invention, the specific method for collecting auxiliary driving information includes the following steps:

[0244] According to the preset maximum speed of the test vehicle and the maximum speed limit of surrounding vehicles, a two-lane, two-way straight highway environment with no central dividing strip is established.

[0245] More specifically, in view of the fact that existing studies rarely consider the lane-changing scenario of a two-way two-lane road, the present invention application embodiment uses UC-win / Road and Tobii-Pro Lab to establish a two-way two-lane road environment and collect driver eye movement data. The total length of the simulated road is about 3.75 kilometers, including a two-lane two-way straight road without a central divider. At the same time, in accordance with Chinese road regulations, the maximum speed limit of the test vehicle is set to 70km / h, and the maximum speed limit of surrounding vehicles is 60km / h.

[0246] The surrounding vehicles within a preset range of the test vehicle are divided into an initial lane leader vehicle, an initial lane following vehicle, a target lane leader vehicle and a target lane following vehicle, and at a timestamp, only one of the initial lane leader vehicle, the initial lane following vehicle, the target lane leader vehicle and the target lane following vehicle is retained.

[0247] More specifically, the simulator data is preprocessed to divide the surrounding vehicles within 200m of the test vehicle into four types: initial lane leader vehicle (Pre), initial lane follower vehicle (Fol), target lane leader vehicle (O_Pre) and target lane follower vehicle (O_Fol). At a timestamp, there is only one Pre, Fol, O_Fol and O_Pre, and redundant surrounding vehicles are deleted.

[0248] The timestamp, vehicle position coordinates, yaw angle, direction, speed, number of lanes, distance to the left boundary, distance to the right boundary, throttle, brake, user conflict, and steering speed characteristics of the test vehicle are obtained; the speed difference, acceleration difference, and gap distance in the three-dimensional vector direction between the surrounding vehicles and the initial lane vehicles of the test vehicle at each timestamp, as well as the gap distance, acceleration difference, and speed difference in the three-dimensional vector direction between the surrounding vehicles and the target lane vehicles are calculated.

[0249] More specifically, the basic characteristics of the test vehicle SV are obtained, including timestamp, vehicle position coordinates, yaw angle, direction, speed, number of lanes, distance to the left boundary, distance to the right boundary, throttle (existence = 1, non-existence = 0), brake (existence = 1, non-existence = 0), conflict with the user (existence = 1, non-existence = 0), and steering speed; the initial lane interaction information (speed difference, acceleration difference, and gap distance between the SV and the initial lane vehicle at X, Y, and Z) and target lane interaction characteristics (gap distance, acceleration difference, and speed difference between the SV and the target lane vehicle at X, Y, and Z) between the surrounding vehicles and the test vehicle are calculated at each timestamp.

[0250] The eye movement coordinates of the driver's eye movement information are collected, and the visual interest area is divided into a left visual interest area, a middle visual interest area and a right visual interest area according to the eye movement behavior, and the left visual interest area, the middle visual interest area and the right visual interest area are expanded to obtain the driver's eye movement characteristics of the fixation event duration and whether there is annotation behavior in the left visual interest area, the middle visual interest area and the right visual interest area.

[0251] More specifically, in addition to the basic eye movement coordinates, the eye gaze data is processed using Tobii-ProLab processing software and the AOI area of ​​interest is divided into three parts: left, middle and right, namely the left side of AOI, the middle side of AOI and the right side of AOI, based on the eye movement behavior. In order to obtain the gaze points of the left and right rearview mirrors and the front area during the lane change process and reduce the influence of the eye tracking device error, each AOI area is enlarged according to the specific situation of the experiment. The driver's eye movement characteristics are obtained: the duration of the gaze event, whether there is annotation behavior on the left, middle and right sides of the AOI area of ​​interest (presence = 1, absence = 0).

[0252] Then, the auxiliary driving information is fused according to a preset time sequence, and features are extracted from the fused data to form a lane change data set.

[0253] In an embodiment of the present invention, the method for fusing the auxiliary driving information according to a preset time sequence may specifically include: in a preset time sequence i, according to the data feature fusion function of the auxiliary driving information:

[0254]

[0255] fusing the lane change information with the eye movement information;

[0256] Among them, the data feature vector of the lane changing information feature matrix A and the data feature vector of the eye movement information feature matrix B are h(i) and q(i) respectively, con(A,B) is the data feature fusion function of the lane changing information feature matrix A and the eye movement information feature matrix B, L is the total number of data, α and β are initialization parameters.

[0257] Finally, the lane-changing data set is subjected to a start marking process, and the marked lane-changing data set is identified according to a lane-changing intention recognition model to execute lane-changing control.

[0258] In an embodiment of the present invention, the method for performing start mark processing on the lane change data set may specifically include the following steps:

[0259] First, the valley value of the vehicle's lateral displacement position during lane change is tracked and the corresponding timestamp is determined. Then, the highest value of the vehicle moving in the opposite direction of the lateral displacement closest to the valley point is searched. Finally, it is searched and determined whether there is a behavior of looking at the left rearview mirror before the highest value closest to the valley point, and the closest point is determined as the starting point.

[0260] More specifically, the method of tracking the valley value of the vehicle lateral displacement position during lane changing and determining the corresponding timestamp may include: tracking and obtaining the lane lateral displacement position D of the nth lane change between the center of the test vehicle and the leftmost lane boundary at the timestamp t during lane changing. n,t The local valley value

[0261] More specifically, the method for searching for the highest value of the vehicle moving in the opposite direction of the lateral displacement that is closest to the valley point may specifically include:

[0262] When the test vehicle moves in the opposite direction of the lateral displacement, the highest point closest to the valley value is searched as t prepare1 ∈(t D -δ); where δ is t prepare1 Less than timestamp t D And any real number greater than 0;

[0263] When searching in the opposite direction from the valley value to the lane side, in the interval (t prepare1 ,t ε ) should ensure that 20 consecutive points are greater than the current point; where t ε For the upward distance t in increasing timestamp prepare1 The timestamp of the 20th most recent point.

[0264] More specifically, the method of searching and judging whether there is a behavior of looking at the left rearview mirror before the highest value closest to the valley point, and determining the point closest to the valley point as the starting point may specifically include:

[0265] Search in t prepare1 Have you ever looked at the left rearview mirror?

[0266] If in t prepare1 Previously, we could search for gaze behavior and mark the closest point as t prepare0, and mark the starting point as t LC =t prepare0 , the time window of the data set is t LC =t prepare0 ;

[0267] If in t prepare1 No gaze behavior was found before, so the starting point is marked as t LC =t prepare1 , the time window of the data set is t0-t prepare1 ;

[0268] Wherein, the behavior of looking at the left rearview mirror is determined by optical visual detection.

[0269] In an embodiment of the present invention, the method for identifying a marked lane change data set according to a lane change intention recognition model may specifically include the following steps:

[0270] S1. Input lane-changing time series features (x n,t ,y n,t ):

[0271]

[0272] Among them, x n,t is the input feature x at timestamp t n ,y n,t is the input feature label y at timestamp t n ,y n ∈{0:LC,1:LK} is the category label of lane-changing intention, LC is the lane-changing label, and LK is the lane-keeping label;

[0273] S2. Position encoding of input features using position embedding PE x n,t :

[0274] PE (pos,2i) =sin(pos / 10000 2i / d model )

[0275] PE (pos,2i) =cos(pos / 10000 2i / d model )

[0276] In the formula, pos represents the position, i represents the dimension, and d model is the dimension of the model;

[0277] For the position offset k between features, PE (pos+k,2i) for:

[0278]

[0279] S3. Multiple different attention head mechanisms are introduced to capture the time series characteristics of lane changes. Each head calculates a set of attention scores, which are merged together to generate the final multi-head attention output; where each head is an independent single-head attention mechanism with different learning parameters;

[0280] S4. Perform global max pooling and regularization on the multi-head attention output;

[0281] The global maximum pooling is as follows: input tensor dimension; for each sample, find the maximum element of each feature channel in the input tensor; obtain a vector of preset length containing the maximum value of each feature channel;

[0282] The regularization simulation is as follows: the dimension of the input tensor; during the training process, for each element in the input tensor, the probability of setting it to 0 is p, and the probability of retaining it is (1-p); generating a binary mask with the same shape as the input tensor, randomly selecting each element; multiplying the elements of the input tensor with the binary mask, setting some elements to 0; obtaining a final filtered output tensor in which some elements are set to 0 and others remain unchanged;

[0283] S5 calculates the likelihood of each type of driving intention, determines the final intention according to the likelihood value of each lane change category, and determines the lane change category with the greatest likelihood as the lane change intention at the current timestamp;

[0284] Among them, the probability of each driving intention It can be calculated by the following formula:

[0285]

[0286] Among them, y n,t is the input feature label y at timestamp t n .

[0287] More specifically, in a specific implementation, each head calculates a set of attention scores, and the method in which the attention scores are merged together to generate the final multi-head attention output may specifically include:

[0288] For each input vector (x n,t ,y n,t ) creates a query vector, a key vector and a value vector; wherein the query vector, the key vector and the value vector are respectively n,t ,y n,t ) is obtained by multiplying the three matrices obtained through training;

[0289] Calculate the intent value for each vector:

[0290]

[0291] Where d k is the dimension of the key vector; Q is the query vector; K is the key vector; T is the lane change duration; v is the value vector;

[0292] Add a multi-head attention mechanism in the self-attention layer, map the query, key, and value matrices through different linear transformations, and concatenate the different matrices together before performing linear transformations:

[0293] MultiHead(Q,K,V)=Concat(head1,…head h )W O

[0294] head i =Attention(QW i Q , KW i K ,VW i V )

[0295] In the formula and h represents the head of attention, and d k =d v =d model / h.

[0296] In an embodiment of the present invention, a perturbed data set x may be generated by perturbing the selected sample. n,t , in the perturbed data set x n,t The models are trained on the same network and the differences between the models are measured by the following objective function:

[0297]

[0298] In the formula, f represents the original model; g represents the simple model; G is the set of simple models, π x Represents the data x' in the new dataset n,t With the original data x n,t The distance between them; Ω(g) represents the model complexity g;

[0299] Perturbation x near the sample n,t , the similarity of samples before and after perturbation is obtained according to the cosine similarity of different types of text samples:

[0300]

[0301] Where D(x,z) is the distance between the perturbed instance and the original instance, and σ is the bandwidth that controls the width of the kernel function;

[0302] The cosine similarity is used as the weight to optimize the objective function through linear regression:

[0303]

[0304] Among them, f(z) is the predicted value of the perturbed sample in the original feature, and the predicted value is used as the target value, which is the predicted value in the interpretable space feature; π x (z) is the similarity between the perturbed instance and the original instance; g(z') is the predicted value in the interpretable original feature space.

[0305] In an optional embodiment of the present invention, in order to evaluate the impact of oncoming traffic on the prediction of the vehicle's lane change intention, the original data set of each driver is divided into two different data sets: a data set containing eye movement features, SV basic features and initial lane interaction features; and a data set containing eye movement features, SV basic features, initial lane interaction features and target lane interaction features. These two different training data sets are used to train different models to evaluate their prediction capabilities.

[0306] In another optional embodiment of the present invention, the prediction results of the test set are compared with the true labels, and then the accuracy, precision, recall and F1 score of the model on the test set are calculated, and on this basis, the effect of the lane change intention recognition model is comprehensively evaluated.

[0307] The calculation formula of accuracy is as follows:

[0308]

[0309] The calculation formula of precision is as follows:

[0310]

[0311] The calculation formula for recall is as follows:

[0312]

[0313] The calculation formula of F1 Score is as follows:

[0314]

[0315] The processor in this embodiment may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above processor may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present invention can be implemented or executed. The above processor may be a microprocessor or the above processor may also be any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiment of the present invention can be directly embodied as being executed by a hardware processor, or executed by a combination of hardware and software modules in the processor. The software module may be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory and completes the steps of the above method in conjunction with its hardware.

[0316] Embodiment 4

[0317] An embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the lane change intention recognition method described above is implemented.

[0318] In summary, the embodiments of the present invention provide a lane change intention recognition method, system and computer storage medium, which perform lane change control by performing data fusion, feature extraction and starting marking processing on the collected auxiliary driving information according to a preset time series. By combining the vehicle characteristic parameters and the driver's eye movement characteristics, the problem of lack of robustness of the existing marking method is overcome. At the same time, compared with the traditional time window marking method (TWL), the recognition time is effectively reduced, and a lane change intention recognition model based on a multi-head attention mechanism is established to obtain higher accuracy, which overcomes the disadvantages of the existing lane change initial point marking method that needs to overcome the time window and other single feature marking methods. This is crucial for the driving assistance system to predict lane change risks and provide timely suggestions, and also provides more reaction time for the driver.

[0319] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described order of actions, because according to the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0320] The above embodiments can be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded or executed on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, a computer, a server or a data center by wired (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or a data center containing one or more available media sets. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0321] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0322] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0323] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0324] In the several embodiments provided by the present invention, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0325] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0326] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0327] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program codes.

[0328] The present invention is described with reference to the flowcharts and / or block diagrams of the methods, apparatuses (devices) and computer program products of the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0329] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0330] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0331] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0332] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for identifying lane change intention, characterized in that: The method comprises: Collecting assisted driving information; wherein the assisted driving information includes lane change information and eye movement information; fusing the auxiliary driving information according to a preset time sequence, and extracting features from the fused data to form a lane change data set; The lane-changing data set is subjected to a start-marking process, and the marked lane-changing data set is identified according to a lane-changing intention recognition model to execute lane-changing control.

2. The method according to claim 1, characterized in that The method of fusing the auxiliary driving information according to a preset time sequence comprises: In the preset time series i, the data feature fusion function of the auxiliary driving information is: fusing the lane change information with the eye movement information; Among them, the data feature vector of the lane changing information feature matrix A and the data feature vector of the eye movement information feature matrix B are h(i) and q(i) respectively, con(A,B) is the data feature fusion function of the lane changing information feature matrix A and the eye movement information feature matrix B, L is the total number of data, α and β are initialization parameters.

3. The method according to claim 1, characterized in that The method of performing a start mark process on the lane change data set comprises: Track the valley value of the vehicle's lateral displacement position during lane change and determine the corresponding timestamp; Search for the highest value of the vehicle moving in the opposite direction of the side shift that is closest to the valley point; Search and determine whether there is a behavior of looking at the left rearview mirror before the highest value closest to the valley point, and determine the point closest to the valley point as the starting point.

4. The method according to claim 2, characterized in that: The method of tracking the valley value of the lateral displacement position of the vehicle during lane change and determining the corresponding timestamp includes: Track and obtain the lane lateral displacement position D of the test vehicle center and the leftmost lane boundary at the time stamp t for the nth lane change n,t The local valley value 5. The method according to claim 2, characterized in that: The method of searching for the highest value of the vehicle moving in the opposite direction of the lateral displacement that is closest to the valley point comprises: When the test vehicle moves in the opposite direction of the lateral displacement, the highest point closest to the valley value is searched as t prepare1 ∈(t D -δ); where δ is t prepare1 Less than timestamp t D And any real number greater than 0; When searching in the opposite direction from the valley value to the lane side, in the interval (t prepare1 ,t ε ) should ensure that 20 consecutive points should be greater than the current point; where t ε For the upward distance t in increasing timestamp prepare1 The timestamp of the 20th most recent point.

6. The method according to claim 2, characterized in that The method of searching and judging whether there is a behavior of looking at the left rearview mirror before the highest value closest to the valley point, and determining the point closest to the valley point as the starting point, comprises: Search in t prepare1 Have you ever looked at the left rearview mirror? If in t prepare1 Previously, we could search for gaze behavior and mark the closest point as t prepare0 , and mark the starting point as t LC =t prepare0 , the time window of the data set is t LC =t prepare0 ; If in t prepare1 No gaze behavior was found before, so the starting point is marked as t LC =t prepare1 , the time window of the data set is t0-t prepare1 ; Wherein, the behavior of looking at the left rearview mirror is determined by optical visual detection.

7. The method according to claim 1, characterized in that The method for identifying the marked lane change data set according to the lane change intention recognition model comprises: S1. Input lane-changing time series features (x n,t ,y n,t ): Among them, x n,t is the input feature x at timestamp t n ,y n,t is the input feature label y at timestamp t n ,y n ∈{0:LC,1:LK} is the category label of lane-changing intention, LC is the lane-changing label, and LK is the lane-keeping label; S2. Position encoding of input features using position embedding PE x n,t : ON (pos,2i) =sin(pos / 10000 2i / d model ) ON (pos,2i) =cos(pos / 10000 2i / d model ) In the formula, pos represents the position, i represents the dimension, and d model is the dimension of the model; For the position offset k between features, PE (pos+k,2i) for: S3. Multiple different attention head mechanisms are introduced to capture the time series characteristics of lane changes. Each head calculates a set of attention scores, which are merged together to generate the final multi-head attention output; where each head is an independent single-head attention mechanism with different learning parameters; S4. Perform global max pooling and regularization on the multi-head attention output; The global maximum pooling is as follows: input tensor dimension; for each sample, find the maximum element of each feature channel in the input tensor; obtain a vector of preset length containing the maximum value of each feature channel; The regularization simulation is as follows: the dimension of the input tensor; during the training process, for each element in the input tensor, the probability of setting it to 0 is p, and the probability of retaining it is (1-p); generating a binary mask with the same shape as the input tensor, randomly selecting each element; multiplying the elements of the input tensor with the binary mask, setting some elements to 0; obtaining a final filtered output tensor in which some elements are set to 0 and others remain unchanged; S5 calculates the likelihood of each type of driving intention, determines the final intention according to the likelihood value of each lane change category, and determines the lane change category with the greatest likelihood as the lane change intention at the current timestamp; Among them, the probability of each driving intention It can be calculated by the following formula: Among them, y n,t is the input feature label y at timestamp t n .

8. The method according to claim 7, characterized in that Each head computes a set of attention scores, which are merged together to generate a final multi-head attention output, the method comprising: For each input vector (x n,t ,y n,t ) creates a query vector, a key vector and a value vector; wherein the query vector, the key vector and the value vector are respectively n,t ,y n,t ) is obtained by multiplying the three matrices obtained through training; Calculate the intent value for each vector: Where d k is the dimension of the key vector; Q is the query vector; K is the key vector; T is the lane change duration; v is the value vector; Add a multi-head attention mechanism in the self-attention layer, map the query, key, and value matrices through different linear transformations, and concatenate the different matrices together before performing linear transformations: MultiHead(Q,K,V)=Concat(head1,…head h )W O head i =Attention(QW i Q ,KW i K ,VW i V ) In the formula and h represents the head of attention, and d k =d v =d model / h.

9. The method according to claim 7, characterized in that: The method further comprises: Generate a perturbed dataset x by perturbing the selected samples n,t , in the perturbed data set x n,t The models are trained on the same network and the differences between the models are measured by the following objective function: In the formula, f represents the original model; g represents the simple model; G is the set of simple models, π x Represents the data x' in the new dataset n,t With the original data x n,t The distance between them; Ω(g) represents the model complexity g; Perturbation x near the sample n,t , the similarity of samples before and after perturbation is obtained according to the cosine similarity of different types of text samples: Where D(x,z) is the distance between the perturbed instance and the original instance, and σ is the bandwidth that controls the width of the kernel function; The cosine similarity is used as the weight to optimize the objective function through linear regression: Among them, f(z) is the predicted value of the perturbed sample in the original feature, and the predicted value is used as the target value, which is the predicted value in the interpretable space feature; π x (z) is the similarity between the perturbed instance and the original instance; g(z') is the predicted value in the interpretable original feature space.

10. A lane change intention recognition system, characterized in that: The system comprises: An acquisition module, used to collect auxiliary driving information; wherein the auxiliary driving information includes lane change information and eye movement information; A processing module, used for fusing the auxiliary driving information according to a preset time sequence and extracting features from the fused data to form a lane-changing data set; The control module is used to perform a start marking process on the lane change data set and identify the marked lane change data set according to the lane change intention recognition model to perform lane change control.

Images