A lane line matching method and system based on deep learning

Abstract

The application discloses a lane line matching method based on deep learning, which comprises the following steps: obtaining historical trajectory data of a target vehicle and historical trajectory data of environment vehicles; obtaining all historical lane line information corresponding to the historical trajectory data of the target vehicle based on the historical trajectory data of the target vehicle; aligning each continuous historical lane line information with the historical trajectory data of each vehicle through coding, and then merging to obtain a sample vector; inputting the sample vector into a constructed deep learning network, training and adjusting the deep learning network, so that the deep learning network outputs a similarity calculation result of each historical lane line information and the historical trajectory data of the target vehicle; when actual lane line matching is performed: obtaining an actual vector based on actual trajectory data of the target vehicle, actual trajectory data of the environment vehicles and actual lane line information after coding; the deep learning network outputs a similarity calculation result based on the input actual vector.

Description

TECHNICAL FIELD

[0001] The present application relates to a vehicle driving track analysis method and system, in particular to a lane line matching method and system. BACKGROUND

[0002] Lane line matching is usually used to match the lane line in the high-precision map corresponding to the position of the vehicle. By lane line matching the position of the vehicle, the behavior, future driving track, etc. of the vehicle can be further analyzed and judged.

[0003] The common lane line matching method at present is to match the distance by combining the position of the current vehicle to find the nearest lane line. In the position where the lane lines are complex and cross each other, such as intersections, the historical information of the track is usually combined to calculate the comprehensive similarity or sequence matching with the passing lane line, so as to perform the best lane line matching.

[0004] In addition, some methods treat lane line matching as a classification problem, extract features of lane lines in the selected range, and finally classify the vehicle track to a certain lane line as the best match through a model.

[0005] However, the lane lines in the general map are directional, and a single position point in the area where multiple lane lines cross, such as intersections, often leads to the matching of the wrong lane line. The similarity, distance or sequence matching judgment combined with the historical track position information often leads to incorrect matching due to the non-uniformity of the lane line point and the track.

[0006] In addition, there is a common problem that the lane line information in the high-precision map is usually structured and extracted, and there are many transition lanes split out in the scene of merging or branching at the turning intersection. These often cause errors in lane similarity or sequence matching in lane line matching, thereby affecting the matching result. SUMMARY

[0007] One of the purposes of the present application is to provide a lane line matching method based on deep learning, which adopts the method of filtering and fusing the historical track passing lane line to obtain the candidate lane line for matching, then encodes the lane line and the vehicle historical track through a deep learning model, then encodes the global map using the lane line encoding, and finally calculates the similarity by combining the track, lane encoding and map encoding to perform accurate lane line matching.

[0008] In order to achieve the above purpose, the present application provides a lane line matching method based on deep learning, which includes the following steps:

[0009] acquire historical trajectory data of the target vehicle and historical trajectory data of environment vehicles around the target vehicle;

[0010] acquire all historical lane line information corresponding to the historical trajectory data of the target vehicle based on the historical trajectory data of the target vehicle;

[0011] align each continuous historical lane line information with the historical trajectory data of the target vehicle and the historical trajectory data of the environment vehicles through encoding, and then combine to obtain a sample vector;

[0012] input the sample vector into the constructed deep learning network, train and tune the deep learning network, so that the deep learning network outputs a similarity calculation result of each historical lane line information and the historical trajectory data of the target vehicle, and the similarity calculation result is used for lane line matching;

[0013] when actual lane line matching is performed:

[0014] acquire actual trajectory data of the target vehicle and actual trajectory data of environment vehicles around the target vehicle;

[0015] acquire all actual lane line information corresponding to the actual trajectory data of the target vehicle based on the actual trajectory data of the target vehicle;

[0016] align each continuous actual lane line information with the actual trajectory data of the target vehicle and the actual trajectory data of the environment vehicles through encoding, and then combine to obtain an actual vector;

[0017] input the actual vector into the trained deep learning network, and the deep learning network outputs a similarity calculation result of each actual lane line information and the actual trajectory data of the target vehicle;

[0018] select the actual lane line information with the highest similarity to the actual trajectory data of the target vehicle as the matched lane line.

[0019] As can be seen, the application encodes vehicle trajectories (including target vehicle trajectories and environment vehicle trajectories) and lane lines to unify trajectories of different length nodes with lane lines, and further ensures that the two kinds of information can be compared in the same feature space from the perspective of encoding features.

[0020] Further, in the deep learning-based lane line matching method, acquiring the historical trajectory data of the target vehicle and the historical trajectory data of the environment vehicles around the target vehicle specifically includes the following steps:

[0021] collect historical driving data of the target vehicle and historical driving data of the environment vehicles;

[0022] According to the time stamp, position information of the same vehicle at different time is acquired as historical trajectory data of the vehicle, so as to obtain the historical trajectory data of the target vehicle and the historical trajectory data of the environment vehicle;

[0023] The actual trajectory data of the target vehicle and the actual trajectory data of the environment vehicle around the target vehicle are acquired, and the actual trajectory data of the target vehicle and the actual trajectory data of the environment vehicle around the target vehicle are acquired.

[0024] The actual driving data of the target vehicle and the actual driving data of the environment vehicle are collected.

[0025] According to the time stamp, position information of the same vehicle at different time is acquired as historical trajectory data of the vehicle, so as to obtain the historical trajectory data of the target vehicle and the historical trajectory data of the environment vehicle.

[0026] Further, in the lane line matching method based on deep learning, the historical trajectory data of the target vehicle is acquired, and all historical lane line information corresponding to the target vehicle is acquired.

[0027] The historical trajectory data of the target vehicle is acquired, and all scattered historical trajectory lane line information within a set range of the target vehicle is acquired.

[0028] The historical trajectory lane line information is fused to obtain a plurality of continuous historical lane line information.

[0029] The actual trajectory data of the target vehicle is acquired, and all actual lane line information corresponding to the target vehicle is acquired.

[0030] The actual trajectory data of the target vehicle is acquired, and all scattered actual trajectory lane line information within a set range of the target vehicle is acquired.

[0031] The actual trajectory lane line information is fused to obtain a plurality of continuous actual lane line information.

[0032] In this embodiment, by fusing the scattered trajectory lane line information, continuous lane line information which can match the trajectory path in consistency is obtained.

[0033] In some more specific embodiments, the scattered trajectory lane line information can be fused based on the lane line topological relationship.

[0034] Further, in the lane line matching method based on deep learning, the deep learning network comprises:

[0035] The first encoder is used for high-dimensional mapping of the input sample vector or actual vector, so as to output trajectory encoding vectors and lane line encoding vectors with consistent size and length.

[0036] a first separation module configured to extract the lane line encoding vector and the trajectory encoding vector of the surrounding vehicle from the input trajectory encoding vector and the lane line encoding vector;

[0037] a map encoder configured to map the lane line encoding vector and the trajectory encoding vector of the surrounding vehicle to output a map vector;

[0038] a first concatenation module configured to fuse the map vector with the trajectory encoding vector and the lane line encoding vector output by the first encoder to output a fused vector;

[0039] a first fully connected layer configured to map the fused vector to a high-dimensional space again to output an intermediate feature vector;

[0040] a second separation module and a second concatenation module configured to separate and fuse the intermediate feature vector again to output a final feature vector;

[0041] a second fully connected layer configured to output a similarity based on the final feature vector;

[0042] wherein during the training process, a triplet loss function of the intermediate feature vector is calculated, and a similarity loss function is calculated based on the similarity output by the second fully connected layer to adjust the parameters of the deep learning network.

[0043] In this embodiment, the historical trajectory approaches the lane line to obtain a plurality of lane line information for matching, and then the lane line and the vehicle historical trajectory are encoded by the deep learning network, and then the lane line encoding is mapped, and finally the similarity calculation is combined with the trajectory encoding, the lane encoding and the map encoding. It can be seen that the map encoding is constructed by the lane line to represent the current map environment, so that the subsequent trajectory lane line similarity calculation can be combined with the current environment information.

[0044] Further, in the deep learning-based lane line matching method, the first encoder adopts a structure of full connection and maximum pooling.

[0045] Further, in the deep learning-based lane line matching method, the map encoder adopts a structure of full connection and average pooling.

[0046] Further, in the deep learning-based lane line matching method, the target function of the deep learning network training is to minimize the weighted sum of the triplet loss function and the similarity loss function.

[0047] Another object of the present application is to provide a deep learning-based lane line matching system, which can realize accurate matching of lane lines.

[0048] In order to achieve the above object, the application further provides a lane line matching system based on deep learning, which comprises:

[0049] a data acquisition module, which acquires historical trajectory data of a target vehicle and historical trajectory data of surrounding environment vehicles of the target vehicle, and acquires all historical lane line information corresponding to the historical trajectory data of the target vehicle based on the historical trajectory data of the target vehicle;

[0050] an encoding module, which aligns each continuous historical lane line information with the historical trajectory data of the target vehicle and the historical trajectory data of the surrounding environment vehicles through encoding, and then merges to obtain a sample vector;

[0051] a deep learning network module, which inputs the sample vector into the deep learning network module, trains and adjusts the deep learning network module, so that the deep learning network module outputs a similarity calculation result of each historical lane line information and the historical trajectory data of the target vehicle, and the similarity calculation result is used for lane line matching;

[0052] when actual lane line matching is performed:

[0053] the data acquisition module acquires actual trajectory data of the target vehicle and actual trajectory data of the surrounding environment vehicles of the target vehicle, and acquires all actual lane line information corresponding to the actual trajectory data of the target vehicle based on the actual trajectory data of the target vehicle;

[0054] the encoding module aligns each continuous actual lane line information with the actual trajectory data of the target vehicle and the actual trajectory data of the surrounding environment vehicles through encoding, and then merges to obtain an actual vector;

[0055] the actual vector is input into the trained deep learning network module, and the deep learning network module outputs a similarity calculation result of each actual lane line information and the actual trajectory data of the target vehicle; the actual lane line information with the highest similarity to the actual trajectory data of the target vehicle is the matched lane line.

[0056] Further, in the lane line matching system based on deep learning, the deep learning network module comprises:

[0057] a first encoder, which performs high-dimensional mapping on the input sample vector or actual vector to output a trajectory encoding vector and a lane line encoding vector with consistent size length;

[0058] a first separation module, which extracts the lane line encoding vector and the trajectory encoding vector of the surrounding environment vehicles from the trajectory encoding vector and the lane line encoding vector input therein;

[0059] a map encoder, which encodes a lane line encoding vector and a trajectory encoding vector of an environment vehicle in an input to output a map vector;

[0060] a first concatenation module, which fuses the map vector with the trajectory encoding vector and the lane line encoding vector output by the first encoder to output a fused vector;

[0061] a first fully connected layer, which maps the fused vector to a high-dimensional space again to output an intermediate feature vector;

[0062] a second separation module and a second concatenation module, which separate and fuse the intermediate feature vector again to output a final feature vector;

[0063] a second fully connected layer, which outputs a similarity based on the final feature vector;

[0064] wherein in the training process, a triplet loss function of the intermediate feature vector is calculated, and a similarity loss function is calculated based on the similarity output by the second fully connected layer to tune the deep learning network.

[0065] Further, in the lane line matching system based on deep learning, the target function of the deep learning network module training is to minimize the weighted sum of the triplet loss function and the similarity loss function.

[0066] The lane line matching method based on deep learning has the following advantages and beneficial effects:

[0067] One of the innovations of the present application is to encode and align the lane line information and the vehicle trajectory information to ensure that they can be compared in the same feature space, and the constructed local map can allow the feature encoding to consider the environmental information to cope with matching in different scenarios.

[0068] Compared with the current direct matching of lane lines in high-precision maps, the present application fuses the scattered trajectory lane line information, so that the discontinuity caused by lane line selection is continuous, and the fused lane line can be better aligned with the trajectory to be matched in consistency.

[0069] The sampled vehicle trajectory is affected by speed and steering, which may cause the sampling distance to be inconsistent with the sampling distance of the lane line in the high-precision map. Similarly, the lane line has different number of sampling points due to the influence of the direction bending property. Therefore, the present application adopts the map feature encoding method to better unify the encoding and ensure the convenience of feature similarity calculation.

[0070] The vehicle trajectory reflects the driving intention of the vehicle, and in different local scenes, similar operations can produce relatively inconsistent features in the position-based trajectory, based on which the application introduces map coding when calculating the similarity, thereby solving this problem.

[0071] The lane line matching system based on deep learning has the beneficial effects described above. BRIEF DESCRIPTION OF DRAWINGS

[0072] Figure 1 The lane line matching method based on deep learning shows the fused lane line obtained in one embodiment.

[0073] Figure 2 The lane line matching method based on deep learning shows the architecture of the deep learning network module used in one embodiment. DETAILED DESCRIPTION

[0074] The lane line matching method and system based on deep learning will be further explained and described below in conjunction with the drawings and specific embodiments, but this explanation and description do not constitute undue limitations on the technical solutions of the application.

[0075] In some embodiments, the lane line matching method based on deep learning includes a training step and an actual lane line matching step.

[0076] The training step includes:

[0077] 100: Obtain the historical trajectory data of the target vehicle and the historical trajectory data of the surrounding environment vehicles of the target vehicle.

[0078] In some more specific embodiments, step 100 can specifically include:

[0079] 110: Collect the historical driving data of the target vehicle and the historical driving data of the surrounding environment vehicles;

[0080] 120: Obtain the position information of the same vehicle at different times as its historical trajectory data according to the time stamp, to obtain the historical trajectory data of the target vehicle and the historical trajectory data of the surrounding environment vehicles;

[0081] In some specific examples, for step 110, the target vehicle historical driving data is collected on the test road, and at the same time, the historical driving data of the surrounding vehicles is detected and identified by the laser radar, camera or similar sensors carried by the target vehicle. At the same time, the global coordinates given by the self-positioning navigation system of the target vehicle can also be combined to obtain the local map of the position of the target vehicle in the high-definition map drawn in advance, which can include the corresponding lane line position, turning, whether it is an intersection, and whether it is controlled.

[0082] In the collection process of the target vehicle historical driving data, it is recommended to collect different driving data in different driving scenarios. At the same time, there is often a long period of target vehicle trajectory in one scenario, so multiple sections of data with intervals but different driving states can be generated for one target vehicle, such as straight driving, turning, and lane changing.

[0083] In addition, in some embodiments, there can be multiple target vehicles in the same scenario at the same time, so that multiple sets of target vehicle historical driving data corresponding thereto can be obtained.

[0084] For step 120, in some more specific embodiments, the collected historical driving data is sorted out to extract the vehicle position corresponding to the time stamp of different vehicle IDs. The position information of the same vehicle at different times can be obtained as its historical trajectory according to the time stamp. In one specific example, 20 frames of data are collected, and the 20 frames of data are taken as the historical trajectory data of the target vehicle.

[0085] 200: Obtain all historical lane line information corresponding to the target vehicle based on the historical trajectory data of the target vehicle.

[0086] In some more specific embodiments, step 200 can specifically include:

[0087] 210: Obtain all scattered historical trajectory lane line information within the set range of the target vehicle based on the historical trajectory data of the target vehicle.

[0088] 220: Fuse the historical trajectory lane line information to obtain a plurality of continuous historical lane line information.

[0089] In the present application, the historical positioning position of the target vehicle can also extract the historical position data of the lane line within the set range around the target vehicle and the related lane description information data. For example, the historical trajectory lane line information existing within the set range can be obtained for the information of each time point position of the target vehicle. The historical trajectory lane line information can include lane line id, lane line position, and lane line index number (which can represent the direction of the lane line).

[0090] Then, the scattered historical trajectory lane line information (point data) is sorted. Points belonging to the same lane line are classified. Due to the speed direction of the historical trajectory and the granularity of the lane line, the points on the same lane line after classification may not be continuous, and the order may not be continuous. Therefore, they can be merged and fused based on the topological relationship of the lane line, for example, the smallest and largest index number is selected, and a complete continuous historical lane line is generated in order, so that all historical lane lines corresponding to the historical trajectory of the target vehicle can be obtained.

[0091] Figure 1 The fused lane line obtained by the deep learning-based lane line matching method according to the application in an embodiment is shown.

[0092] Figure 1 The line P in the figure represents the historical trajectory of the target vehicle, and the remaining lines represent the historical lane lines. Figure 1 As can be seen from the figure, the fused historical lane line is a relatively long lane line, which includes left turn and straight driving in this example. The fused lane line can be better aligned with the historical trajectories of the target vehicle and the surrounding vehicles.

[0093] 300: Align the continuous historical lane line information, the historical trajectory data of the target vehicle, and the historical trajectory data of the surrounding vehicles by encoding, and then merge to obtain a sample vector.

[0094] Usually, a vehicle trajectory is composed of multiple points, so it can be encoded as a sequence of multiple vectors, each of which contains information representing the trajectory at that time.

[0095] For example, for a historical driving trajectory of a target vehicle traj={(x0, y0), (x1, y1), (x2, y2), (x3, y3), (x4, y4), (x5, y5)}, where (x0, y0), (x1, y1), (x2, y2), (x3, y3), (x4, y4), (x5, y5) are five position points at different times arranged in chronological order, traj can be encoded as: [px, py, vx, vy, step, id], as shown in Table 1:

[0096] Table 1.

[0097]

[0098]

[0099] px, py in Table 1 represent position coordinates corresponding to the x-axis and the y-axis, vx and vy represent velocities corresponding to the x-axis and the y-axis, "step" represents a time stage, and id represents the id of the target vehicle.

[0100] Table 1 only exemplarily lists the encoding of one historical driving track of the target vehicle, and other historical driving tracks of the target vehicle can also be encoded in this way or be adaptively modified based on actual needs.

[0101] Similarly, the historical driving tracks of the environment vehicles can also be encoded in this way or in a similar way.

[0102] Similarly, a lane line (including a historical lane line and an actual lane line in the actual measurement process) is also generally composed of multiple points, and therefore can also be encoded as a sequence of multiple vectors, each of which contains information representing the lane line.

[0103] For example, for a lane line lane = {(u0, v0), (u1, v1), (u2, v2), (u3, v3), (u4, v4), (u5, v5)}, it can be encoded as: [pu, pv, du, dv, direction, id], as shown in Table 2:

[0104] Table 2.

[0105] pu pv du dv direction id u1 v1 u1-u0 v1-v0 0 0 u2 v2 u2-u1 v2-v1 1 0 u3 v3 u3-u2 v3-v2 2 0 u4 v4 u4-u3 v4-v3 3 0 u5 v5 u5-u4 v5-v4 4 0

[0106] pu and pv in Table 2 represent the positions of the lane line points, which correspond to px and py of the vehicle driving track, du and dv represent the distances between two points, "direction" represents the direction, and id represents the id of the lane line.

[0107] Thus, the encoded historical track of the target vehicle, the historical track of the environment vehicle, and the historical lane line information are combined as a sample vector, which is input into a subsequent deep learning network in the form of a two-dimensional matrix for training.

[0108] 400: input the sample vector into the constructed deep learning network, and train and adjust the parameters of the deep learning network, so that the deep learning network outputs a similarity calculation result of each historical lane line information and the historical track data of the target vehicle, and the similarity calculation result is used for lane line matching.

[0109] In a specific example, the sample vector data set can be divided into a training set, a validation set, and a test set, and the ratio of the three is 8:1:1. The data distribution of the three is ensured to be as consistent as possible.

[0110] Figure 2The architecture of the deep learning network module employed by the lane line matching method based on deep learning according to the present application in one embodiment is shown.

[0111] As shown in Figure 2 In this preferred embodiment, the constructed deep learning network includes:

[0112] a first encoder (traj / lane enconde) that performs high-dimensional mapping on the input sample vector to output trajectory encoding vectors and lane line encoding vectors with consistent size length;

[0113] a first split module (split[1:]) that extracts lane line encoding vectors and trajectory encoding vectors of surrounding vehicles from the input trajectory encoding vectors and lane line encoding vectors;

[0114] a map encoder (map encode) that performs map encoding on the input lane line encoding vectors and trajectory encoding vectors of surrounding vehicles to output a map vector;

[0115] a first concatenation module (cat) that fuses the map vector with the trajectory encoding vectors and lane line encoding vectors output by the first encoder to output a fusion vector;

[0116] a first fully connected layer (linear) that maps the fusion vector to a high-dimensional space again to output an intermediate feature vector;

[0117] a second split module (split) and a second concatenation module (cat) that separate and fuse the intermediate feature vector again to output a final feature vector;

[0118] a second fully connected layer (linear) that outputs a similarity based on the final feature vector;

[0119] In the training process, a triplet loss of the intermediate feature vector is calculated, and a similarity loss is calculated based on the similarity output by the second fully connected layer to adjust the parameters of the deep learning network.

[0120] In this preferred embodiment, the first encoder (traj / lane enconde) employs a structure of full connection and max pooling, and the map encoder (map encode) employs a structure of full connection and average pooling.

[0121] In some embodiments, the objective function of the deep learning network training is to minimize the weighted sum of the triplet loss and the similarity loss.

[0122] In this embodiment, the sample vector is input to the first encoder (traj / lane enconde), which adopts a structure of full connection + maximum pooling, maps the feature vector to a high dimension, then maximum value pooling is performed on the features representing the same node for upsampling, and finally the encoding vectors of the trajectory (traj) and the lane (lane) with the same size are output. The first split module (split[1:]) extracts the encoding vector of the lane, and the map encode module encodes the lane. The module architecture adopts a structure of full connection + average pooling. The first splicing module fuses the encoded map vector with the previous traj / lane vector, and then maps it to a high-dimensional space again through the first fully connected layer. At this time, the feature vector can be selected by the anchor point, positive and negative samples through the contrast learning method to calculate the triplet loss, so as to obtain a good feature encoding module parameter. At the same time, the traj and lane encoding representing the local map are separated and fused again to map to the similarity.

[0123] The final training objective function is to minimize the weighted sum of the similarity loss and the triplet loss. The two loss functions guarantee that the model can better learn the similarity relationship between the trajectories and the lane lines under different maps from the final mapping similarity and the intermediate encoding similarity. Thus, the training of the model is completed.

[0124] When performing actual lane line matching:

[0125] 10: Obtain actual trajectory data of the target vehicle and actual trajectory data of the surrounding vehicles around the target vehicle.

[0126] In some more specific embodiments, step 10 can specifically include:

[0127] 101: Collect actual driving data of the target vehicle and actual driving data of the surrounding vehicles;

[0128] 102: Obtain position information of the same vehicle at different times as actual trajectory data of the target vehicle and actual trajectory data of the surrounding vehicles according to the time stamp.

[0129] Among them, for step 101, in some specific examples, the actual driving data of the target vehicle is collected, and at the same time, the actual driving data of the surrounding vehicles around the target vehicle is detected and identified through the laser radar, camera or similar sensor carried by the target vehicle. At the same time, the global coordinates given by the self-positioning and navigation system of the target vehicle can also be combined to obtain the local map of the position where the target vehicle is located, which can contain the corresponding lane line position, turning, whether it is an intersection, and whether it is controlled information.

[0130] In addition, in some embodiments, there can be multiple target vehicles in the same scene, and thus multiple sets of actual driving data of the target vehicles can be obtained.

[0131] For step 102, in some more specific embodiments, the collected actual driving data is sorted, and the vehicle positions corresponding to different vehicle IDs and time stamps are extracted. The position information of the same vehicle at different time points is obtained as the actual trajectory of the vehicle. In a specific example, 20 frames of data are collected, and the 20 frames of data are taken as the actual trajectory data of a target vehicle. In addition, the lane line position data and related lane description information data within a set range around the target vehicle can also be extracted based on the vehicle positioning position.

[0132] 20: obtaining all actual lane line information corresponding to the target vehicle based on the actual trajectory data of the target vehicle.

[0133] In some more specific embodiments, step 20 can specifically include:

[0134] 201: obtaining all scattered actual trajectory lane line information within a set range around the target vehicle based on the actual trajectory data of the target vehicle.

[0135] 202: fusing the actual trajectory lane line information to obtain a plurality of continuous actual lane line information.

[0136] In the present application, the actual lane line position data and related lane description information data within a set range around the target vehicle can also be extracted based on the actual positioning position of the target vehicle. For example, the actual trajectory lane line information existing within a set range can be obtained for the information of each time point position of the target vehicle. The actual trajectory lane line information can include lane line id, lane line position, and lane line index number (which can represent the direction of the lane line).

[0137] Then, the scattered actual trajectory lane line information (point data) is sorted. The points belonging to the same lane line are classified. Due to the factors of actual trajectory speed, direction, and lane line granularity, the points on the same lane line after classification can not be continuous, and the order can not be continuous. Therefore, the lane lines can be combined and fused based on the topological relationship of the lane lines, for example, the minimum and maximum index numbers are selected, and a complete continuous actual lane line is generated in order, so that all actual lane lines corresponding to the actual trajectory of the target vehicle can be obtained.

[0138] 30: aligning each continuous actual lane line information with the actual trajectory data of the target vehicle and the actual trajectory data of the environment vehicle through coding, and then merging to obtain an actual vector.

[0139] The specific encoding process is the same as the training phase, and thus will not be described here. Then, the encoded target vehicle actual trajectory, the actual trajectory of the surrounding vehicle, and the actual lane line information can be combined as an actual vector, and input into the deep learning network in the form of a two-dimensional matrix.

[0140] 40: inputting the actual vector into the deep learning network that has been trained, and outputting the similarity calculation results of each actual lane line information and the actual trajectory data of the target vehicle; selecting the actual lane line information with the highest similarity to the actual trajectory data of the target vehicle as the matched lane line.

[0141] Another embodiment of the present application also provides a lane line matching system based on deep learning, which comprises:

[0142] a data acquisition module that acquires historical trajectory data of a target vehicle and historical trajectory data of surrounding vehicles around the target vehicle, and acquires all historical lane line information corresponding to the historical trajectory data of the target vehicle based on the historical trajectory data of the target vehicle;

[0143] an encoding module that aligns each continuous historical lane line information with the historical trajectory data of the target vehicle and the historical trajectory data of the surrounding vehicles through encoding, and then combines them to obtain a sample vector;

[0144] a deep learning network module that inputs the sample vector into the deep learning network module, trains and adjusts the parameters of the deep learning network module, so that the deep learning network module outputs similarity calculation results of each historical lane line information and the historical trajectory data of the target vehicle, which are used for lane line matching;

[0145] When performing actual lane line matching:

[0146] the data acquisition module acquires actual trajectory data of a target vehicle and actual trajectory data of surrounding vehicles around the target vehicle, and acquires all actual lane line information corresponding to the actual trajectory data of the target vehicle based on the actual trajectory data of the target vehicle;

[0147] the encoding module aligns each continuous actual lane line information with the actual trajectory data of the target vehicle and the actual trajectory data of the surrounding vehicles through encoding, and then combines them to obtain an actual vector;

[0148] inputting the actual vector into the deep learning network module that has been trained, and outputting similarity calculation results of each actual lane line information and the actual trajectory data of the target vehicle; the actual lane line information with the highest similarity to the actual trajectory data of the target vehicle is the matched lane line.

[0149] In some more specific embodiments, the deep learning network module comprises: Figure 2 as shown, including:

[0150] a first encoder (traj / lane enconde) for high-dimensional mapping of the input sample vector to output trajectory encoding vectors and lane encoding vectors with consistent size length;

[0151] a first split module (split[1:]) for extracting lane encoding vectors and trajectory encoding vectors of the environment vehicles from the input trajectory encoding vectors and lane encoding vectors;

[0152] a map encoder (map encode) for map encoding of the input lane encoding vectors and trajectory encoding vectors of the environment vehicles to output map vectors;

[0153] a first concatenation module (cat) for fusing the map vectors with the trajectory encoding vectors and lane encoding vectors output by the first encoder to output fusion vectors;

[0154] a first fully connected layer (linear) for re-mapping the fusion vectors to a high-dimensional space to output intermediate feature vectors;

[0155] a second split module (split) and a second concatenation module (cat) for re-separating and fusing the intermediate feature vectors to output final feature vectors;

[0156] a second fully connected layer (linear) for outputting similarity based on the final feature vectors;

[0157] wherein in the training process, a triplet loss of the intermediate feature vectors is calculated, and a similarity loss is calculated based on the similarity output by the second fully connected layer to adjust the parameters of the deep learning network.

[0158] wherein the first encoder (traj / lane enconde) adopts a structure of full connection and maximum pooling, and the map encoder (map encode) adopts a structure of full connection and average pooling.

[0159] In some preferred embodiments, the target function of the deep learning network module training is to minimize the weighted sum of the triplet loss and the similarity loss.

[0160] As can be seen, the present application encodes and aligns the lane line information and the vehicle trajectory information to ensure that they can be compared in the same feature space, and the constructed local map can make the feature encoding consider the environmental information to cope with matching in different scenarios.

[0161] Compared with the current direct matching of the lane line in the high-precision map, the dispersed trajectory lane line information is fused, so that the discontinuity caused by lane line selection is continuous again, and the fused lane line can be aligned with the trajectory to be matched in consistency.

[0162] The map feature coding method adopted in the application can better unify the lane line information and the vehicle trajectory information, and ensure the calculation convenience in feature similarity.

[0163] Since the vehicle trajectory reflects the driving intention of the vehicle, in different local scenes, similar operations may produce inconsistent features in the position-based trajectory, and the application introduces map coding when calculating the similarity, thereby solving the problem.

[0164] It should be noted that the prior art part in the protection scope of the application is not limited to the embodiments given in the application file, all prior art including but not limited to prior patent documents, prior published publications, prior public use, etc. can be included in the protection scope of the application.

[0165] In addition, the combination mode of each technical feature in the case is not limited to the combination mode recorded in the claims of the case or the combination mode recorded in the specific embodiments, all the technical features recorded in the case can be freely combined or combined in any mode, unless contradictory to each other.

[0166] It should be noted that the above-mentioned embodiments are only specific embodiments of the application. Obviously, the application is not limited to the above-mentioned embodiments, and similar changes or modifications can be directly derived or easily conceived by those skilled in the art from the disclosure of the application, and all should belong to the protection scope of the application.

Claims

1. A lane matching method based on deep learning, characterized in that, Including the following steps: Acquire historical trajectory data of the target vehicle and historical trajectory data of environmental vehicles surrounding the target vehicle; Obtain all historical lane line information corresponding to the target vehicle based on its historical trajectory data; The historical lane line information of each consecutive vehicle is aligned with the historical trajectory data of the target vehicle and the historical trajectory data of the surrounding vehicles through encoding, and then merged to obtain a sample vector. The sample vectors are input into the constructed deep learning network, and the deep learning network is trained and its parameters are tuned so that the deep learning network outputs the similarity calculation results between each historical lane line information and the historical trajectory data of the target vehicle. The similarity calculation results are used for lane line matching. When performing actual lane line matching: Acquire the actual trajectory data of the target vehicle and the actual trajectory data of the environmental vehicles surrounding the target vehicle; Obtain all actual lane line information corresponding to the target vehicle based on its actual trajectory data; The actual lane line information of each continuous vehicle is aligned with the actual trajectory data of the target vehicle and the actual trajectory data of the surrounding vehicles through encoding, and then merged to obtain the actual vector. The actual vectors are input into the trained deep learning network, and the deep learning network outputs the similarity calculation results between each actual lane line information and the actual trajectory data of the target vehicle. Select the actual lane line information that has the highest similarity to the actual trajectory data of the target vehicle as the matching lane line; The deep learning network includes: The first encoder performs a high-dimensional mapping on the input sample vector or actual vector to output a trajectory encoding vector and a lane line encoding vector of the same size and length. The first separation module extracts the lane line encoding vector and the trajectory encoding vector of the environment vehicle from the trajectory encoding vector and lane line encoding vector in the input. The map encoder performs map encoding on the input lane line encoding vector and the environmental vehicle trajectory encoding vector to output a map vector. The first stitching module fuses the map vector with the trajectory encoded vector and lane line encoded vector output by the first encoder to output a fused vector; The first fully connected layer maps the fused vector back to a high-dimensional space to output an intermediate feature vector; The second separation module and the second splicing module further separate and merge the intermediate feature vectors to output the final feature vector. The second fully connected layer outputs similarity based on the final feature vector; During training, the triplet loss function of the intermediate feature vectors and the similarity loss function based on the similarity output of the second fully connected layer are calculated to tune the parameters of the deep learning network.

2. The lane matching method based on deep learning as described in claim 1, characterized in that, Obtaining the historical trajectory data of the target vehicle and the surrounding environment includes the following steps: Collect historical driving data of the target vehicle and historical driving data of environmental vehicles; Based on the timestamp, the location information of the same vehicle at different times is obtained as its historical trajectory data, so as to obtain the historical trajectory data of the target vehicle and the historical trajectory data of the environmental vehicle. Obtaining the actual trajectory data of the target vehicle and the surrounding environment includes the following steps: Collect actual driving data of the target vehicle and actual driving data of vehicles in the environment; The location information of the same vehicle at different times is obtained based on the timestamp as its actual trajectory data, so as to obtain the actual trajectory data of the target vehicle and the actual trajectory data of the environmental vehicle.

3. The lane matching method based on deep learning as described in claim 1, characterized in that, The specific steps for obtaining all historical lane line information corresponding to a target vehicle based on its historical trajectory data include: Based on the historical trajectory data of the target vehicle, obtain lane information for all dispersed historical trajectories of the target vehicle within a defined range; The lane line information along the historical trajectory is fused to obtain several consecutive pieces of historical lane line information; The specific steps for obtaining all actual lane line information corresponding to the target vehicle based on its actual trajectory data include: Based on the actual trajectory data of the target vehicle, obtain lane information for all dispersed actual trajectory paths within a defined range of the target vehicle; The lane line information along the actual trajectory is fused to obtain several consecutive pieces of the actual lane line information.

4. The lane matching method based on deep learning as described in claim 1, characterized in that, The first encoder adopts a fully connected and max-pooling structure.

5. The lane matching method based on deep learning as described in claim 1, characterized in that, The map encoder employs a fully connected and average pooling structure.

6. The lane matching method based on deep learning as described in claim 1, characterized in that, The objective function for training the deep learning network is to minimize the weighted sum of the triplet loss function and the similarity loss function.

7. A lane matching system based on deep learning, characterized in that, include: The data acquisition module acquires historical trajectory data of the target vehicle and historical trajectory data of environmental vehicles surrounding the target vehicle; based on the historical trajectory data of the target vehicle, it acquires all historical lane line information corresponding to it. The encoding module aligns the continuous historical lane line information with the historical trajectory data of the target vehicle and the historical trajectory data of the surrounding vehicles through encoding, and then merges them to obtain a sample vector. The deep learning network module is trained and its parameters are tuned by inputting sample vectors into the deep learning network module so that the deep learning network module outputs the similarity calculation results between each historical lane line information and the historical trajectory data of the target vehicle. The similarity calculation results are used for lane line matching. When performing actual lane line matching: The data acquisition module acquires the actual trajectory data of the target vehicle and the actual trajectory data of the surrounding vehicles; based on the actual trajectory data of the target vehicle, it acquires all corresponding actual lane line information. The encoding module aligns the continuous actual lane line information with the actual trajectory data of the target vehicle and the actual trajectory data of the surrounding vehicles through encoding, and then merges them to obtain the actual vector. The actual vectors are input into the trained deep learning network module, and the deep learning network module outputs the similarity calculation results between each actual lane line information and the actual trajectory data of the target vehicle. The actual lane line information with the highest similarity to the actual trajectory data of the target vehicle is the lane line that matches it. The deep learning network includes: The first encoder performs a high-dimensional mapping on the input sample vector or actual vector to output a trajectory encoding vector and a lane line encoding vector of the same size and length. The first separation module extracts the lane line encoding vector and the trajectory encoding vector of the environment vehicle from the trajectory encoding vector and lane line encoding vector in the input. The map encoder performs map encoding on the input lane line encoding vector and the environmental vehicle trajectory encoding vector to output a map vector. The first stitching module fuses the map vector with the trajectory encoded vector and lane line encoded vector output by the first encoder to output a fused vector; The first fully connected layer maps the fused vector back to a high-dimensional space to output an intermediate feature vector; The second separation module and the second splicing module further separate and merge the intermediate feature vectors to output the final feature vector. The second fully connected layer outputs similarity based on the final feature vector; During training, the triplet loss function of the intermediate feature vectors and the similarity loss function based on the similarity output of the second fully connected layer are calculated to tune the parameters of the deep learning network.

8. The lane matching system based on deep learning as described in claim 7, characterized in that, The objective function for training the deep learning network module is to minimize the weighted sum of the triplet loss function and the similarity loss function.

Images