Path recommendation method, model training method, device, equipment and vehicle

Abstract

The embodiment of the invention provides a path recommendation method and device, a model training method and device, equipment and a vehicle. The method comprises the steps that a time sequence data set of a multi-lane scene where a vehicle is located currently is acquired; wherein the time sequence data set comprises driving data of a plurality of driven road sections in a historical time period; each traveled road section is determined based on the complexity of the multi-lane scene; processing the time sequence data set based on a multi-head attention mechanism of a path recommendation model to obtain a transmission fraction matrix; and processing the emission fraction matrix to obtain a current lane guidance recommendation path of the vehicle. According to the method, the lane changing decision safety, the intelligent level and the scene adaptability in the multi-lane scene can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to a path recommendation method, a model training method, an apparatus, a device and a vehicle. BACKGROUND

[0002] In an intelligent transportation system, vehicle lane-changing decision in a multi-lane scenario is a key link of automatic driving technology. For example, in a highway scenario, a vehicle needs to dynamically adjust a driving path in a complex road network of 7 lanes to cope with sudden traffic congestion, construction restrictions or changes in navigation targets (such as right turns, U-turns, etc.).

[0003] In the prior art, a reference path is generated by a predefined traffic rule and a path planning algorithm (such as EM Planner of Apollo), and path and speed planning is performed in a preset coordinate system.

[0004] However, in the above-mentioned manner, fixed rules are relied on, and it is difficult to cope with complex scenarios such as multi-vehicle dynamic interaction and sudden construction, resulting in low lane-changing decision efficiency or increased safety risks. SUMMARY

[0005] Embodiments of the present application provide a path recommendation method, a model training method, an apparatus, a device and a vehicle, which can improve the safety, intelligence level and scenario adaptability of lane-changing decision in a multi-lane scenario.

[0006] In a first aspect, embodiments of the present application provide a path recommendation method, comprising:

[0007] obtaining a time series dataset of a multi-lane scenario in which a vehicle is currently located; wherein the time series dataset comprises driving data of a plurality of traveled road segments in a historical time period; each of the traveled road segments is determined based on the complexity of the multi-lane scenario;

[0008] processing the time series dataset based on a multi-head attention mechanism of a path recommendation model to obtain a firing score matrix; wherein the firing score matrix comprises a firing score of each of the traveled road segments; the firing score represents the possibility of the vehicle being in each lane in the traveled road segment;

[0009] processing the firing score matrix to obtain a lane guide recommendation path of the vehicle at present.

[0010] In a possible implementation, the processing of the time series dataset based on the multi-head attention mechanism of the path recommendation model to obtain the firing score matrix comprises:

[0011] performing feature extraction on the time series dataset to obtain an initial feature vector; wherein the initial feature vector comprises driving feature data of each lane in each of the traveled road segments;

[0012] based on the multi-head attention mechanism, processing the initial feature vector to obtain a global semantic feature; wherein the global semantic feature includes lane semantic features of each of the traveled road segments; the lane semantic features represent lane association relationships between the traveled road segments and each other, and association relationships between each lane in the traveled road segment;

[0013] linearly mapping the global semantic feature to obtain the emission score matrix.

[0014] In a possible implementation, the feature extraction on the time series data set to obtain an initial feature vector includes:

[0015] feature extraction on the time series data set to obtain multi-source time series features; wherein the multi-source time series features include initial driving feature data of each lane in each of the traveled road segments;

[0016] structured processing of the multi-source time series features to obtain a feature matrix;

[0017] mapping of the feature matrix based on an embedding layer of the path recommendation model to obtain the initial feature vector.

[0018] In a possible implementation, the processing of the initial feature vector based on the multi-head attention mechanism to obtain a global semantic feature includes:

[0019] based on the multi-head attention mechanism, determining weight information of each lane in the traveled road segment according to a priority of each lane in the traveled road segment; wherein the weight information includes an attention weight of the lane and an attention weight of driving feature data of the lane;

[0020] attention processing of driving feature data of each lane in each of the traveled road segments according to the weight information of each lane in each of the traveled road segments to obtain the global semantic feature.

[0021] In a possible implementation, the processing of the emission score matrix to obtain the lane guidance recommendation path of the vehicle at present includes:

[0022] based on an improved conditional random field algorithm, processing the emission score matrix and road network topology constraint information of a multi-lane scene where the vehicle is currently located to obtain the lane guidance recommendation path of the vehicle at present.

[0023] In a possible implementation, the improved conditional random field algorithm is used to process the emission score matrix and the road network topology constraint information of the multi-lane scene where the vehicle is currently located, to obtain the current lane guidance recommended path of the vehicle, including:

[0024] According to the improved conditional random field algorithm, the emission score matrix is processed according to the road network topology constraint information, to obtain a topology constraint transition matrix; the topology constraint transition matrix includes transition scores of the current lane under the road network constraint condition; the transition score represents the rationality of the vehicle changing lanes from the current lane to each lane in the next road segment.

[0025] The topology constraint transition matrix and the emission score matrix are decoded and calculated to obtain a local sequence; the local sequence includes lane scores of the current lane under the road network constraint condition; the lane score represents the possibility of the vehicle changing lanes from the current lane to each lane in the next road segment.

[0026] According to the local sequence, the current lane guidance recommended path of the vehicle is generated.

[0027] In a possible implementation, the time sequence data set includes road network topology information, environment information, self-vehicle information, and historical operation information; the road network topology information includes lane attributes, inter-lane connectivity, and road network topology; the environment information includes side-by-side vehicle information and traffic condition information; and the historical operation information includes route trajectories and lane changing information of each traveled road segment.

[0028] In a second aspect, an embodiment of the present application provides a model training method applied to path recommendation, including:

[0029] Obtaining a training data set in a multi-lane scene in a first time period; the training data set includes driving data of a plurality of first road segments; each traveled road segment is determined based on the complexity of the multi-lane scene.

[0030] Processing the training data set based on a multi-head attention mechanism of an initial model to obtain an emission score matrix; the emission score matrix includes emission scores of each first road segment; the emission score represents the possibility of the vehicle being in each lane in the first road segment.

[0031] Processing the emission score matrix to obtain a lane guidance recommended path in a multi-lane scene in a second time period; the second time period is later than the first time period.

[0032] train the initial model based on the lane guidance recommended path to obtain a path recommendation model; wherein the path recommendation model is used to process the time series data set as described in the first aspect to obtain a lane guidance recommended path of the vehicle.

[0033] In a possible implementation, the multi-head attention mechanism based on the initial model is used to process the training data set to obtain a score matrix, including:

[0034] perform feature extraction on the training data set to obtain an initial feature vector; wherein the initial feature vector includes driving feature data of each lane in each first road segment;

[0035] process the initial feature vector based on the multi-head attention mechanism to obtain a global semantic feature; wherein the global semantic feature includes lane semantic features of each first road segment; the lane semantic features represent lane correlation relationships between the first road segment and other first road segments and correlation relationships between lanes in the first road segment;

[0036] perform linear mapping on the global semantic feature to obtain the score matrix.

[0037] In a possible implementation, the feature extraction on the training data set to obtain an initial feature vector includes:

[0038] perform feature extraction on the training data set to obtain multi-source time series features; wherein the multi-source time series features include initial driving feature data of each lane in each first road segment;

[0039] perform structural processing on the multi-source time series features to obtain a feature matrix;

[0040] map the feature matrix based on an embedding layer of the initial model to obtain the initial feature vector.

[0041] In a possible implementation, the processing of the initial feature vector based on the multi-head attention mechanism to obtain a global semantic feature includes:

[0042] determine weight information of each lane in the first road segment based on the multi-head attention mechanism and a priority of each lane in the first road segment; wherein the weight information includes attention weights of the lane and attention weights of driving feature data of the lane;

[0043] perform attention processing on driving feature data of each lane in each first road segment based on the weight information of each lane in each first road segment to obtain the global semantic feature.

[0044] In a possible implementation, the processing of the emission score matrix to obtain the lane guidance recommended path in the multi-lane scene in the second time period comprises:

[0045] The emission score matrix and the road network topology constraint information in the multi-lane scene in the first time period are processed based on an improved conditional random field algorithm to obtain the lane guidance recommended path in the multi-lane scene in the second time period.

[0046] In a possible implementation, the processing of the emission score matrix and the road network topology constraint information in the multi-lane scene in the first time period based on the improved conditional random field algorithm to obtain the lane guidance recommended path in the multi-lane scene in the second time period comprises:

[0047] The emission score matrix is processed based on the improved conditional random field algorithm and the road network topology constraint information to obtain a topology constraint transition matrix, wherein the topology constraint transition matrix comprises a transition score of a current lane under a corresponding road network constraint condition; and the transition score represents rationality of a vehicle changing lanes from the current lane to each lane in a next road segment.

[0048] The topology constraint transition matrix and the emission score matrix are decoded and calculated to obtain a local sequence, wherein the local sequence comprises a lane score of the current lane under the corresponding road network constraint condition; and the lane score represents a possibility of the vehicle changing lanes from the current lane to each lane in the next road segment.

[0049] The lane guidance recommended path in the multi-lane scene in the second time period is generated according to the local sequence.

[0050] In a possible implementation, the time series data set comprises road network topology information, environment information, self-vehicle information, and historical operation information; the road network topology information comprises lane attributes, inter-lane connectivity, and road network topology; the environment information comprises side vehicle information and traffic condition information; and the historical operation information comprises route trajectories and lane changing information of each traveled road segment.

[0051] In a third aspect, an embodiment of the present application provides a path recommendation device, comprising:

[0052] An acquisition module is configured to acquire time series data set of a multi-lane scene in which a vehicle is currently located; wherein the time series data set comprises driving data of a plurality of traveled road segments in a historical time period; each of the traveled road segments is determined based on complexity of the multi-lane scene.

[0053] The processing module is configured to process the time-series data set based on a multi-head attention mechanism of the path recommendation model to obtain an emission score matrix, wherein the emission score matrix comprises an emission score of each of the traveled road segments, and the emission score represents a possibility of the vehicle being in each lane in the traveled road segment.

[0054] The recommendation module is configured to process the emission score matrix to obtain the lane guidance recommendation path of the vehicle at present.

[0055] In a fourth aspect, an embodiment of the present application provides a model training device applied to path recommendation, comprising:

[0056] The acquisition module is configured to acquire a training data set in a multi-lane scene in a first time period, wherein the training data set comprises driving data of a plurality of first road segments, and each of the traveled road segments is determined based on complexity of the multi-lane scene.

[0057] The acquisition module is configured to acquire a training data set in a multi-lane scene in a first time period, wherein the training data set comprises driving data of a plurality of first road segments, and each of the traveled road segments is determined based on complexity of the multi-lane scene.

[0058] The first processing module is configured to process the training data set based on a multi-head attention mechanism of an initial model to obtain an emission score matrix, wherein the emission score matrix comprises an emission score of each of the first road segments, and the emission score represents a possibility of the vehicle being in each lane in the first road segment.

[0059] The second processing module is configured to process the emission score matrix to obtain a lane guidance recommendation path in a multi-lane scene in a second time period, wherein the second time period is later than the first time period.

[0060] The training module is configured to train the initial model based on the lane guidance recommendation path to obtain a path recommendation model, wherein the path recommendation model is configured to process the time-series data set as described in the third aspect to obtain the lane guidance recommendation path of the vehicle.

[0061] In a fifth aspect, an embodiment of the present application provides an electronic device, comprising a memory and a processor.

[0062] The memory stores computer execution instructions.

[0063] The processor executes the computer execution instructions stored in the memory, so that the processor executes the first aspect and / or various possible implementation manners of the first aspect.

[0064] In a sixth aspect, an embodiment of the present application provides a vehicle, the vehicle being provided with a vehicle body and the electronic device according to the fifth aspect.

[0065] In a seventh aspect, an embodiment of the present application provides a computer readable storage medium, the computer readable storage medium storing computer execution instructions, the computer execution instructions being executed by a processor to implement the first aspect and / or various possible implementation manners of the first aspect.

[0066] In an eighth aspect, an embodiment of the present application provides a computer program product, including a computer program, the computer program being executed by a processor to implement the first aspect and / or various possible implementation manners of the first aspect.

[0067] The path recommendation method, model training method, device, equipment and vehicle provided by the embodiments of the present application are used to determine a plurality of traveled road segments based on the complexity of the multi-lane scene in which the vehicle is currently located, process the driving data of each traveled road segment by using the multi-head attention mechanism of the path recommendation model, obtain the possibility of the vehicle being in each lane in each traveled road segment, and obtain the lane guidance recommendation path of the vehicle at present, and then, based on the multi-head attention mechanism of the scene complexity, the lane guidance recommendation path of the vehicle at present is predicted according to the time series data set of the multi-lane scene, so that a safe, efficient and self-adaptive intelligent lane changing decision is realized in a complex road network. BRIEF DESCRIPTION OF DRAWINGS

[0068] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0069] Figure 1 An application scenario schematic diagram provided by the present application;

[0070] Figure 2 A flowchart of a path recommendation method provided by an embodiment of the present application;

[0071] Figure 3 A time series data set schematic diagram provided by an embodiment of the present application;

[0072] Figure 4 A lane guidance recommendation path schematic diagram provided by an embodiment of the present application;

[0073] Figure 5 A flowchart of another path recommendation method provided by an embodiment of the present application;

[0074] Figure 6 A lane transfer probability calculation example schematic diagram provided by an embodiment of the present application;

[0075] Figure 7 A schematic diagram of a lane guidance recommended path generation process provided for an embodiment of the present application;

[0076] Figure 8 A schematic diagram of a feature decoding process based on an improved conditional random field algorithm provided for an embodiment of the present application;

[0077] Figure 9 A flowchart of a model training method applied to path recommendation provided for an embodiment of the present application;

[0078] Figure 10 A flowchart of another model training method applied to path recommendation provided for an embodiment of the present application;

[0079] Figure 11 A flowchart of a sample collection stage provided for an embodiment of the present application;

[0080] Figure 12 A flowchart of a model training stage provided for an embodiment of the present application;

[0081] Figure 13 A flowchart of another model training stage provided for an embodiment of the present application;

[0082] Figure 14 A structural schematic diagram of a path recommendation device provided for an embodiment of the present application;

[0083] Figure 15 A structural schematic diagram of a model training device applied to path recommendation provided for an embodiment of the present application;

[0084] Figure 16 A structural schematic diagram of an electronic device provided for an embodiment of the present application.

[0085] The specific embodiments of the present application have been shown through the above-described drawings, and will be described in more detail hereinafter. These drawings and written descriptions are not intended to limit the scope of the concept of the present application by any means, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0086] The exemplary embodiments will be described in detail herein with reference to the attached drawings. The same reference numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments are not meant to represent all implementations consistent with the present application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the present application as detailed in the appended claims.

[0087] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use, processing, transmission, provision, disclosure and application of related data comply with relevant laws, regulations and standards, necessary security measures are taken, do not violate public order and good customs, and provide corresponding operation portal for user to choose authorization or refusal.

[0088] And the present application involves big data analysis of user information (including but not limited to personal biological characteristics, identity data, consumption data, asset data, electronic terminal operation data, etc.), and uses artificial intelligence technology for automatic decision making, and makes technical solutions based on automatic decision making results that have a significant impact on user rights and interests, provides corresponding operation portal for user to choose to agree or refuse automatic decision making results; if the user chooses to refuse, enter the expert decision making process.

[0089] Figure 1 An application scenario schematic diagram provided by the present application is shown in FIG. 1, the present application is applicable to automatic driving scenarios in multi-lane complex road network, including highway (7 lanes), urban expressway curve (5 lanes) and park road (3 lanes). Figure 1

[0090] According to the above-mentioned scenarios, it can be known that the rule-based method relies on pre-defined logic and cannot cope with sudden multi-car lane changing or road construction; the reinforcement learning-based method produces unsafe behavior (such as crossing multiple lanes on a curve) due to the trial-and-error process, and the reward function is single, making it difficult to model lane-level features.

[0091] The path recommendation method provided by the present application is based on the multi-head attention mechanism of scene complexity, processes the time series data set of multi-lane scene, obtains the current lane guide recommendation path of the vehicle, and realizes safe, efficient and self-adaptive intelligent lane changing decision in complex road network.

[0092] The technical solutions of the present application and how the technical solutions of the present application solve the above-mentioned technical problems will be described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of the present application will be described below with reference to the accompanying drawings.

[0093] Figure 2 A flowchart of a path recommendation method provided by an embodiment of the present application is shown in FIG. 2, the method comprises: Figure 2

[0094] ​​201、obtain a time series dataset of a multi-lane scene where the vehicle is currently located; wherein the time series dataset comprises driving data of a plurality of traveled road segments in a historical time period; each traveled road segment is determined based on complexity of the multi-lane scene.

[0095] Exemplarily, the execution subject of the embodiment can be an electronic device, hereinafter referred to as a device, such as a vehicle-mounted artificial intelligence (AI) chip. The device can be deployed in the cloud or in a vehicle. In a multi-lane scene where the vehicle is currently located, the target driving route is evenly divided into a plurality of road segments with a distance D (D can be adjusted according to the situation, and here D=5m). The device can determine the complexity of the multi-lane scene where the vehicle is currently located according to the scene information (such as scene images) of the current multi-lane scene through a scene perception module, and determine the window length of the data collection window corresponding to the complexity of the multi-lane scene through a preset rule, that is, determine a plurality of traveled road segments that the vehicle has traveled from the current to the past in the window length. The device can obtain a time series dataset of the multi-lane scene where the vehicle is currently located through real-time data collection by sensors in the data collection module, which includes driving data of the plurality of traveled road segments, such as driving state and scene data of the vehicle. For a multi-lane scene with low complexity, such as a straight road segment, the window length is shortened to 10 tokens (covering s-10 to s road segments); for a multi-lane scene with high complexity, such as a curved road or an intersection road segment, the window length is extended to 20 tokens (covering s-20 to s road segments), to capture lane dynamic changes in a longer time dimension (such as the deceleration trend of a car on the side), in order to balance “simple scene efficiency” and “complex scene accuracy”, reduce redundant calculation, and improve inference speed.

[0096] In one example, the time series dataset includes global information and local information. The global information includes road network topology information, specifically including lane attributes (such as whether the current lane is a U-turn, left-turn, straight, or right-turn lane), inter-lane connectivity (such as whether it is a dashed line or a solid line, and whether it can change lanes in that distance), road network topology (such as whether the current lane is a straight or curved lane, the curvature of the curved lane, and the connection relationship with other lanes, etc.).

[0097] Furthermore, local information includes environmental information, vehicle information, and historical operation information. Environmental information includes information about other vehicles (speed and position in surrounding lanes) and traffic conditions such as accidents, obstacles, traffic lights, and police direction; this type of information is collected and provided by LiDAR, high-definition cameras, and millimeter-wave radar. Vehicle information includes vehicle speed and position (current lane conditions, distance to the next intersection, distance to solid lines prohibiting lane changes, etc.). Historical operations include route trajectories and lane change information for each previously traveled segment; for example, recording the route trajectory and lane change information for the previous segment (from ST to S-1) can improve computational efficiency in subsequent processing stages.

[0098] For example, Figure 3 This is a schematic diagram of a time-series dataset provided in an embodiment of this application, as shown below. Figure 3 As shown, in a 7-lane scenario, a continuous road segment is evenly divided into multiple segments with a distance of D (D can be adjusted according to the situation, here we take D=5m). The road network topology, environmental information, vehicle information, historical operations, and other information of each segment are combined into a token (unit). To ensure that the model is adapted to the 7-lane scenario, the tokens of the obtained t consecutive road segments are formed into a time series dataset. The current road segment is represented by s. A time series dataset includes the road network topology information, environmental information, vehicle information, and historical operations of t consecutive road segments. Here, t directly determines the sequence length and can be adjusted according to the actual situation (for the sake of example, we take t=5 here, but in actual cases t will be longer).

[0099] 202. Based on the multi-head attention mechanism of the path recommendation model, the time series dataset is processed to obtain the emission score matrix; wherein, the emission score matrix includes the emission score of each traveled road segment; the emission score represents the probability that a vehicle is in each lane in the traveled road segment.

[0100] For example, Figure 4 This is a schematic diagram of a lane-changing intelligent decision-making process provided in an embodiment of this application, as shown below. Figure 4 As shown, the device pre-trains the model by inputting manually sampled data and large-scale data samples into the model to be trained, using local or cloud computing resources, resulting in a trained model, i.e., a path recommendation model. The device then deploys this path recommendation model locally or in the cloud. After collecting real-time data (time-series dataset) through sensors, the time-series dataset is input into the path recommendation model. Based on the multi-head attention mechanism of this path recommendation model, the time-series dataset is processed to obtain an emission score matrix. This emission score matrix includes the emission score for each traveled road segment, representing the probability that a vehicle is in each lane within that traveled segment.

[0101] For example, the emission score of the s section can be exemplified as: [1.2, 0.8, -0.5, -3.2, 6.8, 2.5, -10], where the 5th lane score is the highest, and the 7th emergency lane score is the lowest, indicating that the vehicle is most likely to be in the 5th lane in the s section.

[0102] 203, processing the emission score matrix to obtain the current lane guidance recommended path of the vehicle.

[0103] Exemplarily, in combination with Figure 4 , the device can utilize the path recommendation module of the path recommendation model, such as a preset prediction mechanism or a Softmax function, to process the possibility of the vehicle being in each lane in each traveled section in the emission score matrix, obtain the possibility of the vehicle being in each lane in the next section, and determine the lane corresponding to the highest possibility as the target lane for recommended lane changing, thereby obtaining the current lane guidance recommended path for intelligent lane changing decision operation.

[0104] It is worth supplementing that after each lane change is completed, the "real-time features + actual lane + lane change result (success / failure)" can be added to the incremental training set, and the path recommendation model can be fine-tuned regularly (e.g., every quarter) to continuously improve the robustness of the path recommendation model.

[0105] In this embodiment, a path recommendation method is provided, the attention mechanism enables the model to accurately capture the matching possibility between each section and each lane, reducing invalid information interference; the multi-source data enables the path recommendation model to adapt to multi-lane scenarios, without the need to reconstruct parameters to cover different road types, reducing deployment costs; and further, safe, efficient and self-adaptive intelligent lane changing decisions are realized in complex road networks, significantly improving the safety and intelligence level of lane changing decisions in multi-lane scenarios.

[0106] Figure 5 Another flowchart of a path recommendation method provided by the embodiments of the present application is shown in Figure 5 , and the method comprises:

[0107] 301, obtaining a time series data set of a multi-lane scenario in which the vehicle is currently located; wherein the time series data set comprises driving data of a plurality of traveled sections in a historical time period; each traveled section is determined based on the complexity of the multi-lane scenario.

[0108] Exemplarily, this step can be referred to as step 201, which will not be described here.

[0109] 302, performing feature extraction on the time series data set to obtain an initial feature vector; wherein the initial feature vector comprises driving feature data of each lane in each traveled section.

[0110] Exemplarily, based on the feature extraction layer of the path recommendation model (for example, a pre-training language model based on the Transformer architecture (Bidirectional Encoder Representations from Transformers, BERT for short)), the driving data (including road network topology information, environmental information, self-vehicle information, and historical operation information) of each traveled road segment in the time series data set is subjected to feature extraction, the driving feature data of each lane in each traveled road segment is obtained, and the driving feature data of each lane in each traveled road segment is vectorized to obtain an initial feature vector.

[0111] In a possible implementation, step 302 includes the following steps: for each traveled road segment, the driving data of each traveled road segment in the time series data set is subjected to preliminary feature extraction to obtain initial driving feature data of each lane in each traveled road segment; wherein the initial driving feature data of each lane can include road network topology dynamic features, environmental features, self-vehicle state features, and historical operation features. The relevant driving features of each traveled road segment are integrated as a token, and every t consecutive road segments, i.e., t tokens, are combined into a sequence, i.e., to obtain multi-source time series features. Through a feature structuring module, the multi-source time series features are subjected to structured processing, for example, matrixing, to obtain a corresponding feature matrix, including the initial driving feature data of each lane in each traveled road segment. Through an embedding layer of the path recommendation model, the feature matrix is mapped to obtain an initial feature vector; for example, a dim-dimensional feature is mapped to a 768-dimensional initial feature vector.

[0112] For example, feature extraction is performed on the time series data set, and feature digitization is performed to obtain multi-source time series features. Among them, for the lane line attribute in the road network topology feature in the multi-source time series feature, each lane has a set of lane lines on both sides, and a set of lane lines consists of two lines. The number of lane line groups is lane number + 1. Among them, the solid line is represented by the number 1, and the dashed line is represented by the number 0. Taking a 7-lane as an example, the lane line group is labeled from left to right as 0 to 7. At this time, the corresponding lane line example data sequence is: [0:1,1;1:0,1;2:0,1;3:0,0;4:0,0;5:0,0;6:1,0;7:1,1]. Taking the first lane and the second lane as an example, the corresponding lane line attribute between the two lanes is [1:0,1], which means that the first lane to the second lane is a dashed line, and the first lane can change lanes to the second lane, but the second lane to the first lane is a solid line, so the second lane cannot change lanes to the first lane. Each lane has two values representing lane changing, the first value represents whether it can change lanes to the left, and the second value represents whether it can change lanes to the right; 0 represents no lane changing, and 1 represents lane changing. For example: [1:0,0;2:0,1;3:1,1;4:1,1;5:1,1;6:1,0;7:0,0]. For the lane attribute in the road network topology feature in the multi-source time series feature, 0 represents keeping the current lane and only straight driving, 1 represents left turning, 2 represents U-turn, and 3 represents right turning. The example data sequence is: [1:1,2;2:1;3:1,2;4:0;5:0;6:0,3;7:3], and the specific content of the example data sequence is shown in Table 1.

[0113] Table 1

[0114]

[0115] For the ego state feature in the multi-source time series feature, each traveled road segment (Token) records the current lane number (1-7), ego speed (km / h), ego acceleration (m / s²), and distance to the front vehicle (m). The example data is: [s-4:3,80,0,80; s-3:3,80,0,70; s-2:3,80,0,60; s-1:3,80,0,50; s:3,80,0,40] (s road segment is 40m away from the front slow-moving vehicle), and the related information is shown in Table 2.

[0116] Table 2

[0117]

[0118] For the environmental features in the multi-source time sequence features, each token records the "vehicle existence (0 = no car, 1 = car)" of the 1-7th lane, the "relative speed of the side car (km / h, negative value indicates that the side car is slower than the ego car)", and the "lane line type (0 = dashed line, 1 = solid line)". The example data (s road section time) is: [1:0,0,0;2:1,-10,0;3:1,-40,1;4:1,0,1;5:0,0,0;6:1,5,0;7:0,0,2] (2 represents an emergency lane and does not be occupied). For the historical operation features in the multi-source time sequence features, including lane transfer probability and historical path sequence, at the initial time, the collected lane transfer probability can be used to initialize the probability of each lane to the surrounding lane. Then, the historical operation path is recorded. For example, assuming that the s-4 road section is the initial time (the vehicle just enters the guided lane change), the current lane transfer probability is [0.0036, 0.0024, 0.0007, 0.9800, 0.0000, 0.0133, 0.0000] (each lane has a corresponding transfer probability, here only the transfer probability of the current lane is given), and the probability of the 4th lane is the highest, that is, the tendency of transferring to the 4th lane is the largest. Then, the historical operation is recorded. Assuming that the historical path sequence recorded from the s-4 road section to the s road section is [s-4:3;s-3:4;s-2:4;s-1:4], that is, the vehicle transfers from the initial 3rd lane to the 4th lane and keeps. For example, based on the premise of the front right turn, based on the historical trajectory of the historical time period, the probability of each lane continuing to straighten, changing the right lane, and changing the left lane under the condition of right turn as the destination in the historical time period is calculated, that is, the lane level transfer probability in the historical time period is obtained. Figure 6 A schematic diagram of a calculation example of a lane transfer probability provided for an embodiment of the present application is shown in Figure 6 As shown, during the 7:00-7:10 time period, the vehicle is in the middle straight lane, and the probability of changing the right lane is 35%, the probability of straightening is 60%, and the probability of changing the left lane is 5% (the surrounding environmental features can be added).

[0119] In one example, the multi-source time-series features are subjected to continuous feature normalization, continuous feature normalization, time-series splicing and other structured processing to convert them into feature vectors that can be understood by the model to obtain a feature matrix. For example, for continuous feature normalization, the speed limit conditions of the road segment are set, the safety distance information is set, and the speed, distance and acceleration are normalized for subsequent data calculation. The Min-Max scaling is used to scale to the interval [0, 1] (the interval can be adjusted according to road regulations and road conditions), and a specific example is: the speed of the ego vehicle on the s road segment is 80 km / h→80 / 120≈0.67, and the distance from the front vehicle is 40 m→40 / 150≈0.27. For continuous feature normalization, an example is: lane number 3→one-hot encoding [0, 0, 1, 0, 0, 0, 0]. For example, for time-series splicing, the t token feature vectors from s-4 to s time are spliced into a time-series feature matrix with a dimension of “[t, dim]” (t is the window length, t=5 in the example, and dim is the single token dimension (a single road segment is a token, which contains all related features of the road segment), and the dimension number can be changed according to the demand), which is used as the model input.

[0120] The structured processing improves the standardization of data. For example, in a 7-lane highway scenario, the structured processed metadata can clearly define the lane attributes and connection relationships, ensuring that the multi-modal feature data contains complete physical rule information, thereby enhancing the modeling capability of the subsequent model for road network topology constraints and avoiding conflicts between decisions and physical rules.

[0121] 303, based on the multi-head attention mechanism, the initial feature vector is processed to obtain global semantic features; wherein the global semantic features include lane semantic features of each traveled road segment; the lane semantic features represent the lane correlation between the traveled road segment and other traveled road segments and the correlation between the lanes in the traveled road segment.

[0122] For example, based on the multi-head attention mechanism of the path recommendation model, the context awareness capability of the multi-head attention mechanism is used to perform semantic analysis and processing on the initial feature vector. For example, based on the self-attention weight coefficient, the driving feature data of each lane in the traveled road segment is subjected to internal attention processing, and / or based on the attention weight coefficient, the driving feature data of each lane is subjected to attention processing between road segments, and finally the global semantic features are obtained, including lane semantic features of each traveled road segment; wherein each lane semantic feature represents the lane correlation between the traveled road segment and other traveled road segments (for example, the speed change of the vehicle in front of the lane 3 of the s-4 to s road segment) and the correlation between the lanes in the traveled road segment (for example, the correlation between the lane 5 of the s road segment without a vehicle and the lane 4 with a vehicle).

[0123] In a possible implementation, step 303 comprises: based on the multi-head attention mechanism, a dynamic attention weight distribution strategy of lane priority can be set, that is, the priority of each lane in each traveled road segment is first determined, and the weight information of each lane in each traveled road segment is distributed according to the priority of each lane in each traveled road segment, including the attention weight of each lane and the attention weight of the driving feature data of each lane. Based on the multi-head attention mechanism, the driving feature data of each lane in each traveled road segment is processed by attention using the weight information of each lane in each traveled road segment, a plurality of attention processing results are obtained, and the plurality of attention processing results are processed by a normalization layer to obtain global semantic features.

[0124] For example, for each driving feature data in the s road segment, the road network topology feature weight accounts for 0.6, the importance of the feature is higher than that of other basic features, and the influence of the feature is greater in calculation; and / or, the environment features (such as vehicle existence, relative speed) of the “adjacent lane of the current lane” and the “target lane” are assigned higher attention weights (accounting for 0.4-0.5), and the “remote lane across more than 3 lanes” feature is assigned a lower attention weight (≤0.1) to reduce invalid feature interference.

[0125] For example, when the ego vehicle is in the 3rd lane, the pre-trained language model (Bidirectional Encoder Representations from Transformers, BERT) based on the Transformer architecture assigns a dynamic weight of 0.45 to the 2nd / 4th lane and a weight of only 0.08 to the 7th lane, so that the emission score is more focused on the “safe lane-changing candidate lane”, and the local decision accuracy is improved.

[0126] The dynamic weight adjustment improves the response ability of the model to key features. For example, in the case of emergency braking, dynamic weight adjustment enables the model to accurately focus on the deceleration trend of the adjacent lane, thereby reducing invalid feature interference and further enhancing decision accuracy in complex scenarios.

[0127] 304, linearly mapping the global semantic features to obtain an emission score matrix.

[0128] Exemplarily, each lane semantic feature in the obtained global semantic feature is linearly mapped by a linear layer (for example, a linear function) of the path recommendation model to obtain an emission score matrix, where the emission score matrix can reflect the "local matching degree" of the current lane to each lane in each road segment, that is, the possibility of the current lane turning to each lane of the next road segment. Through feature processing by the attention mechanism, cross-road and cross-lane feature correlations are obtained respectively, and a more accurate lane-level emission score matrix is finally generated, thereby enhancing the decision accuracy in complex road conditions.

[0129] 305、Based on the improved conditional random field algorithm, the emission score matrix and the road network topology constraint information of the multi-lane scene where the vehicle is currently located are processed to obtain the lane guidance recommendation path of the vehicle at present.

[0130] Exemplarily, Figure 7 A schematic diagram of a lane guidance recommendation path generation process provided by an embodiment of the present application is shown in Figure 7 As shown, the device obtains the road network topology constraint information of the multi-lane scene where the vehicle is currently located, for example, the curved road segment does not allow multi-lane driving. Based on the path recommendation model (i.e., the trained model), the emission score matrix obtained and the road network topology constraint information are processed by using a preset improved conditional random field (CRF) to obtain the lane guidance recommendation path of the vehicle at present, including the target lane of the current road segment and the subsequent path planning; for example, in a 7-lane highway scene, the target lane planning result is "change to No. 5 lane in the current road segment and keep to the next intersection", which ensures that the decision conforms to the physical rules. By dynamically applying the road network topology constraint to improve the CRF algorithm, the conflict between the decision and the physical rules is avoided, and through global semantic modeling and hard constraint embedding, the accuracy and safety of the lane changing decision in complex road conditions are realized, while ensuring that the decision logic is consistent with the historical operation, thereby finally improving the robustness and scene adaptability of the autonomous driving system in the multi-lane scene.

[0131] In a possible implementation, step 305 comprises: based on the improved conditional random field algorithm, performing transition processing on the road network topology constraint information and the emission score corresponding to each lane of the current road segment in the emission score matrix, to obtain a topology constraint transition matrix, including a transition score corresponding to the current lane of the vehicle under the current road network constraint, which represents the rationality of the vehicle changing lanes from the current lane to each lane of the next road segment under the current road network topology constraint; for example, in a curved road scenario, the transition scores of the 3rd lane to the 5th lane are set to -∞, and a reasonable lane (such as the 4th lane) is forced to be selected. Then, the topology constraint transition matrix is used to perform decoding calculation on the emission score of the current road segment of the vehicle in the emission score matrix, to obtain a local sequence, including a lane score of the current lane under the current road network constraint, to represent the possibility of the vehicle changing lanes from the current lane to each lane of the next road segment under the current road network topology constraint. Through the Viterbi algorithm, the lane score of the current lane in the local sequence is recursively, terminated and backtracked, to obtain a continuous lane guidance recommendation path under the current condition, that is, the current lane guidance recommendation path of the vehicle.

[0132] For example, in the stage of constructing the "topology constraint transition matrix", for a straight road segment, the 3rd lane can be transferred to the 2nd / 4th lane (transition score 2), and the score of crossing to the 5th lane is reduced (such as -2 points, so that theoretically the lane changing is allowed, but it is not supported due to safety considerations); for a curved road segment (curvature > 0.05), the 3rd lane can only be transferred to the 4th lane (transition score 2), and cannot be transferred to the 2nd lane (score -10), to avoid dangerous behavior caused by lane changing in a curve; the topology constraint transition matrix can be represented in the form of a matrix [7, 7], where the element S[i][j] represents the rationality score of "transition from the i-th lane to the j-th lane".

[0133] For example, in the stage of constructing the "topology constraint transition matrix", for a straight road segment, the 3rd lane can be transferred to the 2nd / 4th lane (transition score 2), and the score of crossing to the 5th lane is reduced (such as -2 points, so that theoretically the lane changing is allowed, but it is not supported due to safety considerations); for a curved road segment (curvature > 0.05), the 3rd lane can only be transferred to the 4th lane (transition score 2), and cannot be transferred to the 2nd lane (score -10), to avoid dangerous behavior caused by lane changing in a curve; the topology constraint transition matrix can be represented in the form of a matrix [7, 7], where the element S[i][j] represents the rationality score of "transition from the i-th lane to the j-th lane". Figure 8 A schematic diagram of a feature decoding process based on an improved conditional random field algorithm provided by an embodiment of the present application is shown in FIG. 3, which is a schematic diagram of a feature decoding process based on an improved conditional random field algorithm provided by an embodiment of the present application. Figure 8 As shown in FIG. 3, the present scheme only optimizes the s road segment output, and only considers the current s road segment and the subsequent situation during decoding, so that the present scheme is more suitable for the use scenario, and the initial probability needs to be initialized during decoding, and the lanes of the s-4 to s-1 road segments are fixed to a certain value (such as [lane 1, lane 1, lane 2, lane 3]), and do not participate in decoding adjustment; only the transition score of "the 3rd lane of the s-1 road segment to each lane of the s road segment" is calculated, and the local sequence score (such as the 3rd lane to the 4th lane score = emission score + transition score) is obtained by combining the emission score of the s road segment. In the calculation of the transition probability of the current lane, a very small value p is applied to the scheme with a small probability, so that the probability of exploring special paths can be increased to a certain extent, and the possibility of exploring various paths is increased without significantly affecting the original decision.

[0134] By the segmented decoding strategy, the decoding operation is only performed on the current lane of the current road section, ensuring that only the lane transfer possibility of the current road section is optimized, thereby reducing the calculation complexity, while ensuring that the decision is consistent with the historical operation logic, improving the reasoning efficiency and decision reliability.

[0135] In the embodiment, on the basis of the above-mentioned embodiment, on the one hand, the segmented decoding strategy ensures that the decision is consistent with the historical operation logic, improving the reasoning efficiency; on the other hand, by imposing the road network topology constraint, it is ensured that the decision conforms to the physical rules, avoiding illegal lane changing.

[0136] Figure 9 A flowchart of a model training method applied to path recommendation provided by the embodiment of the present application is shown in FIG. 1, which comprises the following steps. Figure 9

[0137] 401. Obtain a training data set in a multi-lane scene in a first time period; wherein the training data set comprises driving data of a plurality of first road sections; each driven road section is determined based on the complexity of the multi-lane scene.

[0138] By way of example, the execution subject of the embodiment can be an electronic device, hereinafter referred to as a device. The device can be deployed in the cloud or in a vehicle. In the multi-lane scene where the vehicle is currently located, the target driving route is evenly divided into a plurality of road sections. The device determines the complexity of the multi-lane scene in a first time period in the past time. Through a preset rule, the window length of the data collection window corresponding to the complexity of the multi-lane scene is determined, i.e., a plurality of driven road sections, i.e., first road sections, in the window length and driven by the vehicle from the current to the past are determined. The device obtains a training data set in a multi-lane scene in a first time period, including driving data in the plurality of first road sections, such as driving state and scene data of the vehicle.

[0139] In one example, the time series data set includes global information and local information. The global information includes road network topology information, specifically including lane attributes, inter-lane connectivity, and road network topology. Further, the local information includes environmental information, vehicle information, and historical operation information. The environmental information includes side vehicle information, and traffic condition information including accidents, obstacles, traffic lights, and traffic police commands, which is collected by laser radar, high-definition camera, millimeter wave radar, etc.; the vehicle information includes vehicle speed and vehicle position; the historical operation includes route trajectory and lane changing information of each driven road section.

[0140] 402. Based on the multi-head attention mechanism of the initial model, the training data set is processed to obtain a transmission score matrix; wherein the transmission score matrix includes the transmission score of each first road section; the transmission score represents the possibility of the vehicle being in each lane in the first road section.​

[0141] Exemplarily, the device calls an initial model. Based on the multi-head attention mechanism (e.g., including self-attention and cross-attention) of the initial model, the training data set is processed to obtain an emission score matrix, including an emission score of each first road segment, to represent the possibility of the vehicle being in each lane in each first road segment, so as to capture the relationship of different lane spaces.

[0142] 403、processing the emission score matrix to obtain a lane guidance recommended path in a multi-lane scene in a second time period; wherein the second time period is later than the first time period.

[0143] Exemplarily, the device can utilize the path recommendation module (e.g., a preset linear layer + Softmax) of the initial model to perform linear mapping processing on the possibility of the vehicle being in each lane in each first road segment in the emission score matrix, to obtain a lane guidance recommended path in a multi-lane scene in a second time period later than the first time period.

[0144] 404、based on the lane guidance recommended path, training the initial model to obtain a path recommendation model; wherein the path recommendation model is used to process the time series data set in the path recommendation method to obtain the lane guidance recommended path of the vehicle.

[0145] Exemplarily, by the lane guidance recommended path and the corresponding actual driving path, a loss function is calculated to adjust the parameters of the initial model to obtain a trained initial model, i.e., a path recommendation model; wherein the path recommendation model is used to process the time series data set in the path recommendation method to obtain the lane guidance recommended path of the vehicle.

[0146] In this embodiment, a model training method applied to path recommendation is provided, which trains the model through structured multi-modal time series data, ensures that the path recommendation model can comprehensively perceive dynamic road network constraints and historical operation logic, thereby providing complete input for subsequent decision-making, obtaining a more accurate lane guidance recommended path, and providing more accurate lane changing decision support.

[0147] Figure 10 Another flowchart of the model training method applied to path recommendation provided by the embodiments of the present application is shown in FIG. 4, which includes the following steps. Figure 10

[0148] 501、obtain a training data set in a multi-lane scene in a first time period; wherein the training data set includes driving data of a plurality of first road segments; each driven road segment is determined based on the complexity of the multi-lane scene.

[0149] Exemplarily, this step can refer to step 401, which will not be described here. ​

[0150] 502. performing feature extraction on the training data set to obtain an initial feature vector; wherein the initial feature vector comprises driving feature data of each lane in each first road segment.

[0151] For example, based on the feature extraction layer of the initial model, the driving data (including road network topology information, environment information, ego vehicle information, and historical operation information) of each first road segment in the training data set is subjected to feature extraction to obtain driving feature data of each lane in each first road segment, and the driving feature data of each lane in each first road segment is vectorized to obtain the initial feature vector.

[0152] In one possible implementation, the step 502 comprises the following steps: performing feature extraction on the training data set to obtain multi-source time series features; wherein the multi-source time series features comprise initial driving feature data of each lane in each first road segment. The multi-source time series features are subjected to structured processing to obtain a feature matrix. The feature matrix is mapped based on the embedding layer of the initial model to obtain the initial feature vector.

[0153] Specifically, Figure 11 A flowchart of the sample collection stage provided by the embodiments of the present application is shown in FIG. 1. Figure 11 As shown in FIG. 1, sample collection is performed in combination with manual sampling and historical big data, and for each first road segment, sample data is obtained, i.e., a data set comprising topology (i.e., road network topology), environment, ego vehicle, and historical operation information is established to obtain a training data set. The driving data (multi-dimensional sample) of each first road segment in the training data set is subjected to digital feature extraction to obtain initial driving feature data of each lane in each first road segment, i.e., a feature tensor; wherein the initial driving feature data of each lane can comprise road network topology dynamic features, environment features, ego vehicle state features, and historical operation features. The relevant driving features of each first road segment are integrated as a token, and every t consecutive road segments, i.e., t tokens, are combined into a sequence to obtain multi-source time series features. Figure 12 A flowchart of the model training stage provided by the embodiments of the present application is shown in FIG. 2. Figure 12 As shown in FIG. 2, after the sample data is subjected to preliminary feature extraction, the multi-source time series features are subjected to structured processing, e.g., matrixing, by a feature structuring module to obtain a corresponding feature matrix comprising initial driving feature data of each lane in each driven road segment. The feature matrix is mapped by the embedding layer of the initial model to obtain the initial feature vector.

[0154] In one example, the initial model is trained in combination with Figure 12In the feature structuring stage, continuous feature normalization, continuous feature normalization, time series splicing and other data preprocessing are performed on the multi-source time series features to convert them into feature vectors that can be understood by the initial model to obtain a feature matrix. The standardization degree of the data is improved through structured processing.

[0155] 503, based on the multi-head attention mechanism, the initial feature vector is processed to obtain global semantic features; wherein the global semantic features include the lane semantic features of each first section; the lane semantic features represent the lane correlation between the first section and other first sections and the correlation between the lanes in the first section.

[0156] Exemplarily, in combination with Figure 12 After obtaining the initial feature vector through feature embedding, based on the multi-head attention mechanism of the initial model (such as BERT model, same as BERT model), the context perception ability of the multi-head attention mechanism is used to perform semantic analysis and processing on the initial feature vector, for example, based on the self-attention weight coefficient, the driving feature data of each lane in each first section is processed by internal attention, and / or based on the attention weight coefficient, the driving feature data of each lane is processed by attention between sections, and finally the global semantic features are obtained, including the lane semantic features of each first section; wherein each lane semantic feature represents the lane correlation between the first section and other first sections (such as the speed change of the vehicle in front of the third lane of the s-5 to s section) and the correlation between the lanes in the first section (such as the correlation between the sixth lane with no car and the seventh lane with car in the s section).

[0157] In one possible implementation, step 503 includes: based on the multi-head attention mechanism, a dynamic attention weight distribution strategy of lane priority can be set, that is, the priority of each lane in each first section is determined first, and the weight information of each lane in each first section is distributed according to the priority of each lane in each first section, including the attention weight of each lane and the attention weight of the driving feature data of each lane. Based on the multi-head attention mechanism, the weight information of each lane in each first section is used to process the driving feature data of each lane in each first section to obtain a plurality of attention processing results, and the global semantic features are obtained by processing the plurality of attention processing results through the normalization layer. The response ability of the model to key features is improved through dynamic weight adjustment. For example, in the case of emergency braking, dynamic weight adjustment enables the model to accurately focus on the deceleration trend of adjacent lanes, thereby reducing invalid feature interference and further enhancing the robustness of the initial model in complex scenarios.

[0158] 504, linearly mapping the global semantic features to obtain a score matrix.

[0159] Exemplarily, in combination with Figure 13 The linear mapping of each lane semantic feature in the obtained global semantic feature is performed through a linear layer (for example, a linear function) of the initial model, to obtain an emission score matrix, wherein the emission score matrix can reflect the relationship information between tokens (i.e., first road segments) and the "local matching degree" between lanes in each token, that is, the possibility of the vehicle being in each lane in the first road segment.

[0160] 505, based on the improved conditional random field algorithm, the emission score matrix and the road network topology constraint information in the multi-lane scene in the first time period are processed to obtain the lane guide recommended path in the multi-lane scene in the second time period.

[0161] Exemplarily, in combination with 12, the device obtains the road network topology constraint information of the multi-lane scene in which the vehicle is currently located, for example, the curved road segment does not allow multi-lane driving. Based on the initial model (i.e., the trained model), the preset improved conditional random field (CRF) algorithm is used to process the obtained emission score matrix and the road network topology constraint information, to obtain the lane guide recommended path in the second time period, including the target lane of the second road segment and the subsequent path planning; for example, in the 6-lane highway scene, the target lane planning result is "change to the 4th lane in the current road segment and keep to the next intersection", to ensure that the decision conforms to the physical rules.

[0162] For example, Figure 13 Another flowchart of the model training stage provided by the embodiment of the present application is as follows: Figure 14As shown, the continuous road segment is divided into fixed-distance units token, the road network topology (lane attribute, connection relationship), environmental information (side car speed, lane line type), vehicle state (speed, acceleration) and historical operation (intended lane changing path) are integrated into a multi-modal feature vector to obtain a structured time sequence feature matrix. BERT calculates the feature correlation across time and across lanes through the multi-head attention mechanism, and assigns dynamic weights (such as giving higher weights to the environmental features of adjacent lanes). Then, the improved CRF generates a transition matrix combining the road network topology constraints, optimizes only the current road segment output (such as s road segment), fixes the historical operation sequence (such as s-4 to s-1 road segment lane), and decodes the optimal path through the Viterbi algorithm. The output of the current road segment is the optimal lane probability distribution and global path planning, for example, in a curve scenario, BERT outputs the emission score of lane 5 as the highest, and CRF combines the topology constraints (such as lane 3 can only be transferred to lane 4) to correct the final decision of lane 4. In the initial stage of training, the model weights are frozen, and the transition probabilities of CRF are partially initialized using historical mining information, until the later weights are fine-tuned. In the process of learning, the mask of the current road segment is gradually removed as the vehicle moves, combining the globally known road network topology information and the locally known vehicle, environment and historical operation information, learning the rules to reasonably constrain the subsequent path prediction.

[0163] By dynamically applying road network topology constraints to improve CRF, learning the matching relationship between different tokens, and embedding global semantic modeling and hard constraints, the robustness and scene adaptability of the initial model in multi-lane scenarios are improved.

[0164] In a possible implementation, step 505 includes: based on the improved conditional random field algorithm, performing calculation and processing on the road network topology constraint information and the current lane to obtain a topology constraint transition matrix, including a transition score corresponding to the current lane, which represents the rationality of the vehicle changing lanes from the current lane to each lane in the next road segment under the corresponding road network topology constraint; for example, in a curved road scenario, the transition score from lane 3 to lane 5 is set to -∞, and a reasonable lane (such as lane 4) is forced to be selected. Then, the emission score of the current lane in the emission score matrix is decoded and calculated by using the topology constraint transition matrix to obtain a local sequence, including the lane score of the current lane under the corresponding road network constraint condition; the lane score represents the possibility of the vehicle changing lanes from the current lane to each lane in the next road segment. The lane score of the current lane in the local sequence is processed by using the Viterbi algorithm to obtain an optimal lane-changing path of each second road segment in the second time period, and the optimal lane-changing path in each second road segment in the second time period is integrated to obtain a continuous lane guidance recommendation path under the corresponding road network constraint condition in the second time period, that is, the lane guidance recommendation path is obtained. By using the segmented decoding strategy, only the current lane of the current road segment is decoded, which ensures that only the lane transition possibility of the current road segment is optimized, thereby reducing the calculation complexity, and ensuring that the decision is consistent with the historical operation logic and improving the inference efficiency and decision reliability of the initial model.

[0165] 506. Train the initial model based on the lane guidance recommendation path to obtain a path recommendation model; wherein the path recommendation model is used to process the time series data set in the path recommendation method to obtain the lane guidance recommendation path of the vehicle.

[0166] For example, this step can refer to step 404, which will not be described here

[0167] In this embodiment, on the basis of the above-mentioned embodiment, on the one hand, the dynamic attention mechanism enables the model to accurately capture the correlation features of adjacent lanes and target lanes, and reduces invalid information interference; on the other hand, the multi-source feature structure enables the trained model to adapt to multi-lane scenarios, without the need to reconstruct parameters to cover different road types, thereby reducing the model deployment cost.

[0168] Figure 14 A structural schematic diagram of a path recommendation device provided in the embodiment of the application is shown in FIG. 1, which includes: Figure 15

[0169] The acquisition module 601 is configured to acquire a time series data set of a multi-lane scenario currently traveled by a vehicle; wherein the time series data set includes driving data of a plurality of traveled road segments in a historical time period; each traveled road segment is determined based on the complexity of the multi-lane scenario.

[0170] ​The processing module 602 is configured to process the time-series data set based on a multi-head attention mechanism of the path recommendation model to obtain an emission score matrix, wherein the emission score matrix comprises an emission score of each traveled road segment, and the emission score represents a possibility of the vehicle being in each lane in the traveled road segment.

[0171] The recommendation module 603 is configured to process the emission score matrix to obtain a lane guide recommendation path of the vehicle at present.

[0172] In a possible implementation, the processing module 602 is specifically configured to: perform feature extraction on the time-series data set to obtain an initial feature vector, wherein the initial feature vector comprises driving feature data of each lane in each traveled road segment; perform processing on the initial feature vector based on the multi-head attention mechanism to obtain global semantic features, wherein the global semantic features comprise lane semantic features of each traveled road segment, and the lane semantic features represent a lane correlation relationship between the traveled road segment and other traveled road segments and a correlation relationship between the lanes in the traveled road segment; and perform linear mapping on the global semantic features to obtain the emission score matrix.

[0173] In a possible implementation, the processing module 602 is specifically configured to: perform feature extraction on the time-series data set to obtain multi-source time-series features, wherein the multi-source time-series features comprise initial driving feature data of each lane in each traveled road segment; perform structured processing on the multi-source time-series features to obtain a feature matrix; and perform mapping on the feature matrix based on an embedding layer of the path recommendation model to obtain the initial feature vector.

[0174] In a possible implementation, the processing module 602 is specifically configured to: determine weight information of each lane in each traveled road segment based on a priority of each lane in the traveled road segment according to the multi-head attention mechanism, wherein the weight information comprises an attention weight of the lane and an attention weight of driving feature data of the lane; and perform attention processing on the driving feature data of each lane in each traveled road segment according to the weight information of each lane in each traveled road segment to obtain the global semantic features.

[0175] In a possible implementation, the recommendation module 603 is specifically configured to: perform processing on the emission score matrix and road network topology constraint information of a multi-lane scene in which the vehicle is currently located based on an improved conditional random field algorithm to obtain the lane guide recommendation path of the vehicle at present.

[0176] In a possible implementation, the recommendation module 603 is specifically configured to: based on an improved conditional random field algorithm, process the emission score matrix according to the road network topology constraint information to obtain a topology constraint transition matrix; the topology constraint transition matrix includes transition scores of a current lane under a road network constraint condition; the transition score represents rationality of a vehicle changing lanes from the current lane to each lane in a next road segment; and the topology constraint transition matrix and the emission score matrix are decoded and calculated to obtain a local sequence; the local sequence includes lane scores of the current lane under the road network constraint condition; the lane score represents a possibility of the vehicle changing lanes from the current lane to each lane in the next road segment; and the lane guide recommendation path of the vehicle at present is generated according to the local sequence.

[0177] In a possible implementation, the time series data set includes road network topology information, environment information, self-vehicle information and historical operation information; the road network topology information includes lane attributes, inter-lane connectivity and road network topology; the environment information includes side vehicle information and traffic condition information; and the historical operation information includes route trajectories and lane changing information of each traveled road segment.

[0178] The device of the embodiment can execute the technical solutions in the above method, and the specific implementation process and technical principles are the same, which will not be described here again.

[0179] Figure 15 A structural schematic diagram of a model training device applied to path recommendation provided by the embodiment of the application is shown in FIG. 1, which includes: Figure 16

[0180] The acquisition module 701 is configured to acquire a training data set in a multi-lane scene in a first time period; the training data set includes driving data of a plurality of first road segments; each traveled road segment is determined based on complexity of the multi-lane scene.

[0181] The first processing module 702 is configured to process the training data set based on a multi-head attention mechanism of an initial model to obtain an emission score matrix; the emission score matrix includes emission scores of each first road segment; the emission score represents a possibility of a vehicle being in each lane in the first road segment.

[0182] The second processing module 703 is configured to process the emission score matrix to obtain a lane guide recommendation path in the multi-lane scene in a second time period; the second time period is later than the first time period.

[0183] The training module 704 is configured to train the initial model based on the lane guide recommendation path to obtain a path recommendation model; the path recommendation model is used to process a time series data set in a path recommendation device to obtain a lane guide recommendation path of a vehicle.

[0184] ​In a possible implementation, the first processing module 702 is specifically configured to: perform feature extraction on the training data set to obtain an initial feature vector; wherein the initial feature vector includes driving feature data of each lane in each first road segment; perform processing on the initial feature vector based on a multi-head attention mechanism to obtain global semantic features; wherein the global semantic features include lane semantic features of each first road segment; the lane semantic features represent lane association relationships between the first road segment and other first road segments and association relationships between the lanes in the first road segment; and perform linear mapping on the global semantic features to obtain an emission score matrix.

[0185] In a possible implementation, the first processing module 702 is specifically configured to: perform feature extraction on the training data set to obtain multi-source time sequence features; wherein the multi-source time sequence features include initial driving feature data of each lane in each first road segment; perform structured processing on the multi-source time sequence features to obtain a feature matrix; and perform mapping on the feature matrix based on an embedding layer of an initial model to obtain an initial feature vector.

[0186] In a possible implementation, the first processing module 702 is specifically configured to: determine weight information of each lane in a first road segment based on a multi-head attention mechanism and a priority of each lane in the first road segment; wherein the weight information includes attention weights of the lanes and attention weights of driving feature data of the lanes; and perform attention processing on driving feature data of each lane in each first road segment based on the weight information of each lane in each first road segment to obtain global semantic features.

[0187] In a possible implementation, the second processing module 703 is specifically configured to: perform processing on the emission score matrix and road network topology constraint information in a multi-lane scene in a first time period based on an improved conditional random field algorithm to obtain a lane guide recommended path in a multi-lane scene in a second time period.

[0188] In a possible implementation, the second processing module 703 is specifically configured to: perform processing on the emission score matrix based on an improved conditional random field algorithm and based on road network topology constraint information to obtain a topology constraint transition matrix; wherein the topology constraint transition matrix includes transition scores of a current lane under a corresponding road network constraint condition; the transition scores represent rationality of a vehicle changing lanes from the current lane to lanes in a next road segment; and perform decoding calculation on the topology constraint transition matrix and the emission score matrix to obtain a local sequence; wherein the local sequence includes lane scores of the current lane under the corresponding road network constraint condition; the lane scores represent a possibility of the vehicle changing lanes from the current lane to the lanes in the next road segment; and generate a lane guide recommended path in a multi-lane scene in a second time period based on the local sequence.

[0189] In a possible implementation, the time-series dataset includes road network topology information, environment information, ego vehicle information, and historical operation information; the road network topology information includes lane attributes, inter-lane connectivity, and road network topology; the environment information includes side vehicle information and traffic condition information; and the historical operation information includes route trajectories and lane changing information of each traveled road segment.

[0190] The device of the embodiment can execute the technical solutions in the above method, and the specific implementation process and technical principles are the same, which will not be repeated here.

[0191] Figure 16 A structural schematic diagram of an electronic device provided by the embodiment is shown in FIG. 8, which includes a memory 801 and a processor 802. The memory 801 is configured to store executable instructions of the processor 802. The processor 802 is configured to execute the method provided by the above embodiment. The electronic device further includes a receiver 803 and a transmitter 804. The receiver 803 is configured to receive instructions and data sent by other devices, and the transmitter 804 is configured to send instructions and data to external devices. The specific implementation process of the processor can refer to the above method embodiment, and the implementation principle and technical effect are similar, which will not be repeated here. ​

[0192] The embodiment of the present application provides a vehicle, which is provided with a vehicle body and an electronic device as in the above embodiment.

[0193] The embodiment of the present application further provides a computer readable storage medium, which stores computer executable instructions. When the computer executable instructions run on the computer, the computer executes the technical solutions of the above embodiment.

[0194] The above readable storage medium can be realized by any type of volatile or non-volatile storage device or their combination, such as static random access memory, electrically erasable programmable read-only memory, erasable programmable read-only memory, programmable read-only memory, read-only memory, magnetic storage, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special purpose computer.

[0195] An exemplary readable storage medium is coupled to the processor, so that the processor can read information from the readable storage medium and write information to the readable storage medium. The readable storage medium can also be an integral part of the processor. The processor and the readable storage medium can be located in a special integrated circuit. The processor and the readable storage medium can also exist as discrete components in the device.

[0196] ​The embodiment of the present application further provides a computer program product, which comprises a computer program stored in a computer readable storage medium, at least one processor can read the computer program from the computer readable storage medium, and the at least one processor executes the computer program to implement the technical solutions in the above embodiment.

[0197] Those skilled in the art can understand that all or part of the steps of the above-mentioned method embodiments can be completed by a program instruction related hardware. The foregoing program can be stored in a computer readable storage medium. The program executes to perform the steps of the above-mentioned method embodiments; and the foregoing storage medium includes various media capable of storing program codes, such as a magnetic disk or an optical disk.

[0198] Finally, it should be noted that: those skilled in the art will readily understand other embodiments of the present application after considering the specification and practicing the application disclosed herein. The present application is intended to cover any variations, uses, or adaptations of the application following the general principles thereof and including the known or customary technical means or common knowledge in the art not disclosed in the present application, and is not limited to the precise structures described above and shown in the drawings, and various modifications and changes can be made without departing from the scope thereof. The scope of the present application is only limited by the appended claims.

Claims

1. A path recommendation method, characterized in that, include: Obtain a time-series dataset of the multi-lane scenario in which the vehicle is currently located; wherein, the time-series dataset includes driving data of multiple driven road segments within a historical time period; each driven road segment is determined based on the complexity of the multi-lane scenario; Based on the multi-head attention mechanism of the path recommendation model, the time-series dataset is processed to obtain an emission score matrix; wherein, the emission score matrix includes the emission score of each of the traveled road segments; the emission score represents the probability that a vehicle is in each lane of the traveled road segment; The emission score matrix is ​​processed to obtain the current lane guidance recommendation path for the vehicle.

2. The method according to claim 1, characterized in that, The multi-head attention mechanism based on the path recommendation model processes the time-series dataset to obtain an emission score matrix, including: Feature extraction is performed on the time-series dataset to obtain an initial feature vector; wherein, the initial feature vector includes driving feature data of each lane in each of the traveled road segments; Based on the multi-head attention mechanism, the initial feature vector is processed to obtain global semantic features; wherein, the global semantic features include lane semantic features of each of the driven road segments; the lane semantic features represent the lane association relationship between the driven road segments and other driven road segments, as well as the association relationship between each lane within the driven road segment; The emission score matrix is ​​obtained by linearly mapping the global semantic features.

3. The method according to claim 2, characterized in that, The step of extracting features from the time-series dataset to obtain an initial feature vector includes: Feature extraction is performed on the time-series dataset to obtain multi-source time-series features; wherein, the multi-source time-series features include the initial driving feature data of each lane in each of the traveled road segments; The multi-source time-series features are structured to obtain a feature matrix; Based on the embedding layer of the path recommendation model, the feature matrix is ​​mapped to obtain the initial feature vector.

4. The method according to claim 2, characterized in that, The process of processing the initial feature vector based on the multi-head attention mechanism to obtain global semantic features includes: Based on the multi-head attention mechanism, the weight information of each lane in the already driven road segment is determined according to the priority of each lane; wherein, the weight information includes the attention weight of the lane and the attention weight of the driving feature data of the lane. Based on the weight information of each lane in each of the traveled road segments, attention processing is performed on the driving feature data of each lane in each of the traveled road segments to obtain the global semantic features.

5. The method according to claim 1, characterized in that, The process of processing the emission score matrix to obtain the vehicle's current lane guidance recommendation path includes: Based on the improved conditional random field algorithm, the emission score matrix and the road network topology constraint information of the current multi-lane scenario of the vehicle are processed to obtain the current lane guidance recommendation path of the vehicle.

6. The method according to claim 5, characterized in that, The improved conditional random field algorithm processes the emission fraction matrix and the road network topology constraint information of the multi-lane scenario in which the vehicle is currently located to obtain the current lane guidance recommendation path for the vehicle, including: Based on the improved conditional random field algorithm, the emission score matrix is ​​processed according to the road network topology constraint information to obtain the topology constraint transition matrix; wherein, the topology constraint transition matrix includes the transition score of the current lane under the road network constraint conditions; the transition score characterizes the rationality of the vehicle changing lanes from the current lane to each lane in the next road segment; The topology constraint transition matrix and the emission score matrix are decoded and calculated to obtain a local sequence; wherein, the local sequence includes the lane score of the current lane under road network constraints; the lane score represents the probability of a vehicle changing lanes from the current lane to various lanes in the next road segment; Based on the local sequence, the current lane guidance recommendation path for the vehicle is generated.

7. The method according to any one of claims 1-6, characterized in that, The time-series dataset includes road network topology information, environmental information, vehicle information, and historical operation information; the road network topology information includes lane attributes, inter-lane connectivity, and road network topology; the environmental information includes adjacent vehicle information and traffic condition information; and the historical operation information includes route trajectories and lane change information for each traveled road segment.

8. A model training method for path recommendation, characterized in that, include: Obtain a training dataset for a multi-lane scenario within a first time period; wherein the training dataset includes driving data for multiple first road segments; each driven road segment is determined based on the complexity of the multi-lane scenario; Based on the multi-head attention mechanism of the initial model, the training dataset is processed to obtain an emission score matrix; wherein, the emission score matrix includes the emission score of each first road segment; the emission score represents the probability that a vehicle is in each lane in the first road segment; The emission score matrix is ​​processed to obtain the lane guidance recommendation path in the multi-lane scenario during the second time period; wherein the second time period is later than the first time period. Based on the lane guidance recommendation path, the initial model is trained to obtain a path recommendation model; wherein, the path recommendation model is used to process the time series dataset as described in any one of claims 1-7 to obtain the lane guidance recommendation path for the vehicle.

9. The method according to claim 8, characterized in that, The multi-head attention mechanism based on the initial model processes the training dataset to obtain an emission score matrix, including: Feature extraction is performed on the training dataset to obtain an initial feature vector; wherein, the initial feature vector includes driving feature data of each lane in each of the first road segments; Based on the multi-head attention mechanism, the initial feature vector is processed to obtain global semantic features; wherein, the global semantic features include lane semantic features of each first road segment; the lane semantic features represent the lane association relationship between the first road segment and other first road segments, as well as the association relationship between each lane within the first road segment; The emission score matrix is ​​obtained by linearly mapping the global semantic features.

10. The method according to claim 9, characterized in that, The step of extracting features from the training dataset to obtain an initial feature vector includes: Feature extraction is performed on the training dataset to obtain multi-source time-series features; wherein, the multi-source time-series features include the initial driving feature data of each lane in each of the first road segments; The multi-source time-series features are structured to obtain a feature matrix; Based on the embedding layer of the initial model, the feature matrix is ​​mapped to obtain the initial feature vector.

11. The method according to claim 9, characterized in that, The process of processing the initial feature vector based on the multi-head attention mechanism to obtain global semantic features includes: Based on the multi-head attention mechanism, the weight information of each lane in the first road segment is determined according to the priority of each lane in the first road segment; wherein, the weight information includes the attention weight of the lane and the attention weight of the driving feature data of the lane. Based on the weight information of each lane in each of the first road segments, attention processing is performed on the driving feature data of each lane in each of the first road segments to obtain the global semantic features.

12. The method according to claim 8, characterized in that, The process of processing the emission score matrix to obtain the lane guidance recommendation path in the multi-lane scenario within the second time period includes: Based on the improved conditional random field algorithm, the emission score matrix and the road network topology constraint information in the multi-lane scenario in the first time period are processed to obtain the lane guidance recommendation path in the multi-lane scenario in the second time period.

13. The method according to claim 12, characterized in that, The improved conditional random field algorithm processes the emission score matrix and the road network topology constraint information in the multi-lane scenario within the first time period to obtain the lane guidance recommendation path in the multi-lane scenario within the second time period, including: Based on the improved conditional random field algorithm, the emission score matrix is ​​processed according to the road network topology constraint information to obtain a topology constraint transition matrix; wherein, the topology constraint transition matrix includes the transition score of the current lane under the corresponding road network constraint conditions; the transition score characterizes the rationality of the vehicle changing lanes from the current lane to each lane in the next road segment; The topology constraint transition matrix and the emission score matrix are decoded and calculated to obtain a local sequence; wherein, the local sequence includes the lane score of the current lane under the corresponding road network constraint conditions; the lane score represents the probability of a vehicle changing lanes from the current lane to various lanes in the next road segment; Based on the local sequence, a lane guidance recommendation path is generated for the multi-lane scenario within the second time period.

14. The method according to any one of claims 8-13, characterized in that, The time-series dataset includes road network topology information, environmental information, vehicle information, and historical operation information; the road network topology information includes lane attributes, inter-lane connectivity, and road network topology; the environmental information includes adjacent vehicle information and traffic condition information; and the historical operation information includes route trajectories and lane change information for each traveled road segment.

15. A route recommendation device, characterized in that, include: The acquisition module is used to acquire a time-series dataset of the multi-lane scenario in which the vehicle is currently located; wherein, the time-series dataset includes driving data of multiple driven road segments within a historical time period; each driven road segment is determined based on the complexity of the multi-lane scenario; The processing module is used to process the time-series dataset based on the multi-head attention mechanism of the path recommendation model to obtain an emission score matrix; wherein, the emission score matrix includes the emission score of each of the traveled road segments; the emission score represents the probability that a vehicle is in each lane in the traveled road segment; The recommendation module is used to process the emission score matrix to obtain the current lane guidance recommendation path for the vehicle.

16. A model training device for path recommendation, characterized in that, include: The acquisition module is used to acquire a training dataset for a multi-lane scenario within a first time period; wherein the training dataset includes driving data for multiple first road segments; each driven road segment is determined based on the complexity of the multi-lane scenario; The first processing module is used to process the training dataset based on the multi-head attention mechanism of the initial model to obtain an emission score matrix; wherein, the emission score matrix includes the emission score of each of the first road segments; the emission score represents the probability that a vehicle is in each lane in the first road segment; The second processing module is used to process the emission score matrix to obtain the lane guidance recommendation path in the multi-lane scenario within the second time period; wherein the second time period is later than the first time period. The training module is used to train the initial model based on the lane guidance recommendation path to obtain a path recommendation model; wherein the path recommendation model is used to process the time series dataset as described in claim 15 to obtain the lane guidance recommendation path for the vehicle.

17. An electronic device, characterized in that, include: Memory, processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1-14.

18. A vehicle, characterized in that, The vehicle is provided with a vehicle body and the electronic equipment as described in claim 17.

19. A computer-readable storage medium / computer program product, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-14; and / or, the computer program product includes a computer program, which, when executed by a processor, implements the method as described in any one of claims 1-14.

Images