Autonomous taxi cruise path and speed planning method and device

Abstract

The application discloses an automatic driving taxi cruise path and speed planning method and device, and the path planning method comprises the following steps: obtaining a historical cruise path, and dividing the historical cruise path to obtain a plurality of combinations comprising a training set and a test set; learning a preset path reward learning model to obtain a preliminary path reward by using road network data; calculating a cruise strategy; selecting a training set of any combination, learning a first expected path access frequency of the current training set based on the cruise strategy; updating the preset path reward learning model according to the first expected path access frequency and a corresponding first actual path access frequency; learning a second expected path access frequency of each test set based on the cruise strategy, and combining the corresponding second actual path access frequency to calculate a coefficient of the preliminary planning reward; combining the preliminary planning reward based on the coefficient to obtain a comprehensive planning reward; and determining an optimal cruise path according to the final path reward. The application has less required data and better effect.

Description

TECHNICAL FIELD

[0001] The present application relates to path planning technology, in particular to an automatic driving taxi cruising path and speed planning method and device. BACKGROUND

[0002] In urban traffic, taxis play an important role, and automatic driving taxis are rapidly entering human daily life. Automatic driving taxis can shorten passenger waiting time, reduce traffic congestion, reduce greenhouse gas emissions, and improve service quality and operational efficiency. In fact, taxis usually spend 35-60% of their working time cruising for potential passengers, which reflects the importance of optimizing taxi cruising paths. Experienced taxi drivers usually plan cruising paths based on personal experience and perform better than novices in planning cruising paths. For automatic driving taxis, how to draw on the driving experience of human taxi drivers and design the best cruising path has attracted the attention of many researchers.

[0003] In recent years, more and more research has focused on taxi cruising path planning, and cruising path planning for a single taxi can be broadly divided into three categories: the first category is to recommend one or more areas (such as grids, regions, road clusters) to an empty car, but not to specify detailed paths; the second category is to recommend a detailed cruising path (i.e. a series of connected road segments) to an empty car and limit the distance or time; the third category is to derive the optimal routing strategy to maximize long-term expected profits. Although these studies have achieved good performance in taxi cruising path planning, there are still two limitations: first, previous studies require a large number of accurate pick-up / drop-off records to mine pick-up locations and trip fees for predicting pick-up probabilities or generating reward functions. Second, these studies ignore cruising speed when planning the cruising path of an automatic driving taxi, which does not fully exploit the advantages of automatic driving and is not effective. SUMMARY

[0004] The present application aims to solve the problems existing in the prior art, and provides an automatic driving taxi cruising path and speed planning method and device which requires less learned data and has better effect.

[0005] Technical solution: The automatic driving taxi cruising path planning method provided by the present application comprises the following steps:

[0006] (1) Obtain historical cruising paths of a plurality of experienced taxi drivers, and divide them according to a preset rule to obtain a plurality of combinations including a training set and a test set;

[0007] (2) Learning a preset path reward learning model on road network data to obtain a preliminary path reward;

[0008] (3) calculating a cruise strategy based on the preliminary path reward and road network conditions;

[0009] (4) selecting any one of the combined training sets, and learning a first expected path access frequency of the current training set based on the cruise strategy;

[0010] (5) updating the preset path reward learning model according to the first expected path access frequency and a corresponding first actual path access frequency;

[0011] (6) judging whether the combination is traversed, if yes, executing step (7), otherwise returning to step (2);

[0012] (7) for each preliminary planning reward, learning a second expected path access frequency of the corresponding test set based on the cruise strategy, and calculating a coefficient of the preliminary planning reward in combination with a corresponding second actual path access frequency;

[0013] (8) linearly combining all the preliminary planning rewards based on the coefficient to obtain a comprehensive planning reward;

[0014] (9) determining an optimal cruise path according to the final path reward.

[0015] Further, the preset rule in step (1) is specifically:

[0016] all historical cruise paths are taken as a basic data set Ω, a part of the basic data set Ω is randomly selected, and the selected cruise paths are taken as a training set T of the current combination i , and the remaining cruise paths are taken as a corresponding test set V i , and the selection is performed n times to obtain n combinations, i represents a combination number, wherein T1≠T2...≠T i ...≠T n , and V1≠V2...V i ...≠V n .

[0017] Further, step (2) specifically includes:

[0018] (2-1) obtaining road network data, the road network data is represented by a directed graph G=(V,E), V and E are respectively a set of road segments and intersections;

[0019] (2-2) inputting the road network data into the preset path reward learning model to obtain a preliminary path reward, the preset path reward learning model specifically includes sequentially connected one-hot encoding layer, first full connection layer, second full connection layer, first normalization layer, third full connection layer, second normalization layer and fourth full connection layer, and the preliminary path reward includes a reward of each road segment in the road.

[0020] Further, step (3) specifically comprises:

[0021] (3-1) generating transition probabilities of each road segment to other road segments according to road network conditions;

[0022] (3-2) setting initial values of values of all road segments in the road network V0(s) = -∞, s∈S, S being a set of all road segments in the road network;

[0023] (3-3) setting iteration number t = 1;

[0024] (3-4) calculating predicted rewards of each road segment in the current iteration based on the preliminary path reward according to the following formula:

[0025]

[0026] wherein Q t (s,a) represents the predicted reward of road segment s performing action a, r i (s) represents the reward of road segment s in the preliminary path reward, γ represents a discount factor, P(s'|s,a) represents the probability of entering road segment s' from road segment s performing action a, and A represents a set of driving actions;

[0027] (3-5) updating the value of each road segment according to the following formula:

[0028]

[0029] V t (s) represents the value of road segment s in the tthiteration;

[0030] (3-6) determining whether t reaches a preset iteration number T, if yes, performing step (3-7), otherwise setting t = t + 1 and returning to perform step (3-4);

[0031] (3-7) calculating a cruising strategy according to the predicted reward value Q T (s,a) and the value V T (s) of each road segment at the end of iteration:

[0032] π(a|s) = exp(Q T (s,a) - V T (s)), s∈S, a∈A

[0033] π = {π(a|s)|s∈S, a∈A}

[0034] wherein π represents the cruising strategy, and π(a|s) represents the probability of road segment s performing action a.

[0035] Further, step (4) specifically comprises:

[0036] (4-1)select any combination of training set T i , from which the maximum path length U is obtained;

[0037] (4-2)from the training set T i , retrieve the number of paths with length 1 starting from road segment s, and assign it to the initial value E of the expected visit frequency 1,s,1 (s), s∈S, S is the set of all road segments in the road network;

[0038] (4-3)according to the cruise strategy, iteratively calculate the expected visit frequency according to the following formula:

[0039] E l+1,s,: (s) = ∑ s′∈S ∑ a∈A P(s|s′,a)π(a|s′)E l,s′,: (s′), s∈S, l = 1, …, U

[0040] In the formula, E l+1,s,: (s) represents the expected visit frequency of road segment s at the lth iteration, E l+1,s,: (s) = {E l+1,s,1 (s), …, E l+1,s,U (s)}, P(s|s′,) represents the probability of entering road segment s from road segment s′ performing action a, A represents the set of driving actions, and π(a|s′) represents the probability of performing action a in the cruise strategy. s′;

[0041] (4-4)according to the expected visit frequency, the first expected path visit frequency is calculated according to the following formula:

[0042]

[0043] In the formula, E l,s,u (s) represents the u-th element in E l,s,: (s), and μ exp [T i ] represents the first expected path visit frequency.

[0044] Further, step (5) specifically comprises:

[0045] (5-1)calculate the first actual path visit frequency according to the training set;

[0046] (5-2)according to the first expected path visit frequency and the first actual path visit frequency, the regularization derivative of the current preset path reward learning model is calculated according to the following formula:

[0047]

[0048] In the formula, denotes a regularization derivative of the preset path reward learning model, L denotes an objective function of the preset path reward learning model, and φ i denotes a network parameter of the current preset path reward learning model, and μ exp denotes a first expected path visit frequency, and μ i denotes a first actual path visit frequency, real i denotes a first actual path visit frequency, denotes a derivative of the preliminary path reward with respect to φ i , denotes a regularization term, and L φ denotes a regularization target value;

[0049] (5-3) updating the network parameter of the preset path reward learning model according to the regularization derivative in a back propagation manner.

[0050] Further, the step (7) specifically comprises:

[0051] (7-1) for each preliminary planning reward, obtaining a corresponding test set thereof;

[0052] (7-2) obtaining a second expected path visit frequency of each test set according to the cruise strategy learning;

[0053] (7-3) calculating a second actual path visit frequency of each test set;

[0054] (7-4) updating a weight formula according to the second expected path visit frequency and the second actual path visit frequency according to the following formula:

[0055]

[0056]

[0057]

[0058] In the formula, e i,j denotes a learning error rate, denote the second expected path visit frequency and the second actual path visit frequency of a road segment j in a test set, respectively, m denotes a total number of road segments in the test set, and w i,j , w i+1,j denote weights of the preset path reward learning model corresponding to the road segment j in the i-th and the i+1-th preliminary planning rewards, respectively, and η is a preset threshold, a i,j denotes a coefficient of the reward of the road segment j in the i-th preliminary planning reward;

[0059] (7-5) calculating a coefficient of each preliminary planning reward according to the following formula:

[0060] ​

[0061]

[0062]

[0063] In the formula, a i represents the coefficient of the i-th preliminary planning reward.

[0064] The automatic driving taxi cruising speed planning method provided by the application comprises the following steps:

[0065] (1) planning a path according to the automatic driving taxi cruising path planning method to obtain an optimal planning path;

[0066] (2) inputting the optimal planning path into a preset self-attention Bi-LSTM model to obtain the speed of each section in the optimal planning path, wherein the preset self-attention Bi-LSTM model comprises a mapping layer, a Bi-LSTM network, a self-attention layer, a first full connection layer, a random inactivation layer and a second full connection layer connected in sequence.

[0067] The automatic driving taxi cruising path planning device provided by the application comprises:

[0068] one or more processors;

[0069] a memory for storing one or more programs;

[0070] When the one or more programs are executed by the one or more processors, the one or more processors implement the automatic driving taxi cruising path planning method described above.

[0071] The automatic driving taxi cruising speed planning device provided by the application comprises:

[0072] one or more processors;

[0073] a memory for storing one or more programs;

[0074] When the one or more programs are executed by the one or more processors, the one or more processors implement the automatic driving taxi cruising speed planning method described above.

[0075] Beneficial effects: Compared with the prior art, the present application has the following advantages: the autonomous taxi cruise path and speed planning technology of the present application is based on deep inverse reinforcement learning, compared with the existing method which needs to rely on accurate pick-up / drop-off data and pays little attention to cruise speed planning, the method proposed in the present application combines IRL with ensemble learning, which reduces the difficulty of data collection while improving the learning effect. In addition, the present application also provides a path-based self-attention bidirectional LSTM model for adjusting the cruise speed on different roads. BRIEF DESCRIPTION OF DRAWINGS

[0076] Figure 1 A flowchart of the autonomous taxi cruise path planning method proposed in the present application is shown in the figure.

[0077] Figure 2 A network structure diagram of the preset path reward learning model proposed in the present application is shown in the figure.

[0078] Figure 3 A structure diagram of the self-attention Bi-LSTM model proposed in the present application is shown in the figure.

[0079] Figure 4 A cruise path planning and cruise speed planning case implemented by the present application is shown in the figure.

[0080] Figure 5 A structure diagram of the autonomous taxi cruise path planning device proposed in the present application is shown in the figure. DETAILED DESCRIPTION

[0081] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0082] Embodiment one

[0083] The present embodiment provides an autonomous taxi cruise path planning method, as shown in the figure, comprising the following steps: Figure 1

[0084] (1) Obtain the historical cruise paths of a plurality of taxi experienced drivers, and divide them according to a preset rule to obtain a plurality of combinations including a training set and a test set.

[0085] ​Experienced taxi drivers generate cruise routes rich in information. Even if a taxi driver doesn't find passengers along the route, his / her choice of passing through these locations based on experience reflects the likelihood that passengers might be waiting at those locations. The cruise route selections of experienced taxi drivers are valuable lessons for autonomous taxis.

[0086] The pre-defined rule for partitioning is as follows: all historical cruise paths are used as the base dataset Ω, and a portion is randomly selected from the base dataset Ω. The selected cruise paths are used as the training set T for the current combination. i The remaining cruise path serves as the corresponding test set V. i A total of n selections are made, resulting in n combinations, where i represents the combination index, i.e., Ω = T1∪V1 = T2∪V2 = ... = T i ∪V i Where T1≠T2...≠T i ...≠T n And V1≠V2...V i ...≠V n .

[0087] (2) The road network data is learned using a preset path reward learning model to obtain preliminary path rewards.

[0088] This step specifically includes:

[0089] (2-1) Obtain road network data, which is represented by a directed graph G = (V, E), where V and E are sets of road segments and intersections, respectively.

[0090] (2-2) Input the road network data into the preset path reward learning model to obtain the initial path reward.

[0091] Among them, such as Figure 2 As shown, the preset path reward learning model specifically includes a one-hot encoding layer, a first fully connected layer, a second fully connected layer, a first normalization layer, a third fully connected layer, a second normalization layer, and a fourth fully connected layer connected in sequence. The network parameters of the preset path reward learning model are updated during iteration. Therefore, the preset path reward learning model used for learning the i-th combination during the i-th training can be called the preset path reward learning model i. The initial path reward includes the reward for each segment of the road, wherein the path reward output by the preset path reward learning model i is R. i =(r i,1 ,r i,2 ,...,r i,m ), r i,m The reward for the m-th path output by the learning model i with a preset path reward, where m is the number of paths in the road network, and r i,mThe range is (-1, +1).

[0092] (3) Cruise strategy is calculated based on the preliminary path reward and road network condition.

[0093] This step specifically includes:

[0094] (3-1) Transition probability of each road segment to other road segments is generated according to road network condition; specifically, it is obtained according to road network road segment adjacency relationship;

[0095] (3-2) Initial value V0(s) = -∞ of value of all road segments in the road network is set, s∈S, S is a set of all road segments in the road network;

[0096] (3-3) Iteration number t = 1 is set;

[0097] (3-4) Prediction reward of each road segment in the current iteration is calculated based on the preliminary path reward according to the following formula:

[0098] Q t (s,a)=r i (s)+γ∑ s′∈S P(s′|s,a)·V t-1 (s′),s∈S,a∈A

[0099] In the formula, Q t (s,a) represents the prediction reward of the road segment s performing the action a, r i (s) represents the reward of the road segment s in the preliminary path reward, γ represents a discount factor, P(s′|s,a) represents the probability of entering the road segment s′ from the road segment s performing the action a, and A represents a set of driving actions, for example, selecting to enter a left-turn downstream road segment, a straight-ahead downstream road segment, a right-turn downstream road segment, etc.

[0100] (3-5) Value of each road segment is updated according to the following formula:

[0101] V t (s)=log∑ a∈A expQ t (s,a)

[0102] V t (s) represents the value of the road segment s in the tth iteration;

[0103] (3-6) It is judged whether t reaches a preset iteration number T, if yes, step (3-7) is performed, otherwise t = t + 1 is set and step (3-4) is returned to be performed;

[0104] (3-7) Cruise strategy is generated according to the prediction reward value Q T (s,a) and the value V T(s), the cruise strategy is calculated:

[0105] π(a|s) = exp(Q T (s, a)-V T (s)), s∈S, a∈A

[0106] π = {π(a|s)|s∈S, a∈A}

[0107] In the formula, π represents the cruise strategy, and π(a|s) represents the probability of performing action a on road segment s.

[0108] (4) Select any combination of training sets, and learn the first expected path access frequency of the current training set based on the cruise strategy.

[0109] The step specifically comprises:

[0110] (4-1) Select any combination of training sets T i , and obtain the maximum path length

[0111] (4-2) Retrieve the number of paths with a length of 1 starting from road segment s from the training set T i , and assign it to the expected access frequency initial value E 1,s,1 (s), s∈S, S is the set of all road segments in the road network;

[0112] (4-3) According to the cruise strategy, the expected access frequency is calculated iteratively according to the following formula:

[0113] E l+1,s,: (s) = ∑ s′∈S ∑ a∈A P(s|s′,a)π(a|s′)E l,s′,: (s′), s∈S, l = 1, …, U

[0114] In the formula, E l+1,s,: (s) represents the expected access frequency of road segment s at the lth iteration, E l+1,s,: (s) = {E l+1,s,1 (s), …, E l+1,s,U (s)}, P(s|s′,a) represents the probability of entering road segment s from road segment s′ by performing action a, A represents the set of driving actions, and π(a|s′) represents the probability of performing action a on road segment s′ in the cruise strategy.

[0115] (4-4) According to the expected access frequency E l,s,: (s), the first expected path access frequency is calculated according to the following formula:

[0116]

[0117] In the formula, E l,s,u (s) represents E l,s,: The u-th element in (s), μ exp [T i ] represents the first expected path access frequency.

[0118] The above process of expected path access frequency considers the inequality of path length. Existing ME-IRL research ignores the inequality of path length when calculating the expected path access frequency. Ignoring the inequality of training path length will cause calculation error of the expected path access frequency, and the present application can effectively solve this problem.

[0119] (5) Update the preset path reward learning model according to the first expected path access frequency and the corresponding first actual path access frequency.

[0120] This step specifically includes:

[0121] (5-1) Calculate the first actual path access frequency according to the training set; when calculating, first split each path in the training set into a road segment, then count the access times of each road segment, and accumulate to obtain the first actual path access frequency;

[0122] (5-2) According to the first expected path access frequency and the first actual path access frequency, the regularization derivative of the current preset path reward learning model is calculated according to the following formula:

[0123]

[0124] In the formula, The regularization derivative of the preset path reward learning model is represented by L, which represents the objective function of the preset path reward learning model, and φ i represents the network parameters of the current preset path reward learning model, μ exp [T i ] represents the first expected path access frequency, μ real [T i ] represents the first actual path access frequency, represents the derivative of the preliminary path reward with respect to φ i , L represents the regularization term, and L φ represents the regularization target value.

[0125] (5-3) Update the network parameters of the preset path reward learning model according to the back propagation of the regularization derivative.

[0126] (6) Determine whether the combination is traversed, if yes, execute step (7), otherwise return to step (2).

[0127] (7) For each preliminary planning reward, a second expected path visit frequency of a corresponding test set is obtained based on the cruise strategy learning, and a coefficient of the preliminary planning reward is calculated in combination with a corresponding second actual path visit frequency.

[0128] This step specifically includes:

[0129] (7-1) For each preliminary planning reward, a corresponding test set is obtained; for example, a preliminary planning reward R i The corresponding test set is V i ;

[0130] (7-2) A second expected path visit frequency of each test set is obtained according to the cruise strategy learning;

[0131] (7-3) A second actual path visit frequency of each test set is calculated;

[0132] (7-4) The weight formula is updated according to the second expected path visit frequency and the second actual path visit frequency according to the following formula:

[0133]

[0134]

[0135]

[0136] In the formula, i,j indicates a learning error rate of a preset path reward learning model i on a road segment s j , m indicates a total number of road segments in the test set, w i,j and w i+1,j respectively indicate weights of road segment j in the preset path reward learning model corresponding to the i-th and i+1-th preliminary planning rewards, and the initial values are both 1 / m; η is a preset threshold, a i,j indicates a coefficient of the reward of road segment j in the i-th preliminary planning reward; and respectively indicate a second expected path visit frequency and a second actual path visit frequency of road segment j in the test set, The calculation formula is as follows:

[0137]

[0138] In the formula, indicates a predicted visit frequency of road segment s j at the l-th iteration, the value indicates a number of paths with a length of 1 starting from road segment s i retrieved from the test set V * , and S * indicates a road segment set in the test set V i ; and the number of iterations is ended Value is assigned to

[0139] (7-5) The coefficient of each preliminary planning reward is calculated according to the following formula:

[0140]

[0141]

[0142]

[0143] In the formula, a i represents the coefficient of the i th preliminary planning reward.

[0144] Unlike the traditional AdaBoost algorithm, the traditional AdaBoost algorithm calculates the learning error rate κ i of each weak learner on the training set, while the present application calculates κ i of each preset path reward learning model on the test set, which can make the evaluation of the preset path reward learning model performance more accurate.

[0145] (8) Linearly combine all the preliminary planning rewards based on the coefficients to obtain a comprehensive planning reward.

[0146] The linear combination method is specifically shown in the following formula:

[0147]

[0148] After combination, a strong learner is formed, and the result is more accurate.

[0149] (9) Determine the optimal cruise path according to the final path reward.

[0150] According to the reward function of each section, the value iteration algorithm is used to obtain the driving path corresponding to the optimal driving strategy.

[0151] Example Two

[0152] Experienced taxi drivers are good at adjusting the cruise speed according to different roads. For example, reduce the cruise speed in hot areas, and increase the cruise speed in cold areas. For novice taxi drivers, it may be difficult to search for passengers on different roads at the recommended cruise speed, which will increase their workload. However, an autonomous taxi can easily adjust the cruise speed on different roads to increase the likelihood of finding waiting passengers or reduce the cruise time. After learning to plan a cruise path from the cruise path of an experienced taxi driver, this embodiment further learns how to control the cruise speed from the path of the experienced taxi driver, so this embodiment provides an autonomous taxi cruise speed planning method, including the following steps:

[0153] (1) Plan the path according to the method of embodiment one to obtain the optimal planning path.

[0154] (2) Input the optimal planning path into the preset self-attention Bi-LSTM model to obtain the speed of each road segment in the optimal planning path.

[0155] Wherein, as shown in Figure 3 The preset self-attention Bi-LSTM model includes a mapping layer, a Bi-LSTM network, a self-attention layer, a first full connection layer, a random inactivation layer, and a second full connection layer connected in sequence.

[0156] Considering that the cruise speed is not only determined by the taxi driver, but also affected by the real-time traffic conditions. Therefore, this embodiment analyzes the ratio of the cruise speed to the real-time normal driving speed (i.e. the average driving speed of the taxi with passengers) to eliminate the influence of each other. Since the output of the Bi-LSTM network is the ratio of the expected driving speed on each road to the real-time normal driving speed, rather than the specified driving speed, the autonomous taxi can flexibly adjust the cruise speed according to the real-time road conditions.

[0157] Considering that each cruise path and path is a sequence of road segments arranged in time sequence, this embodiment uses a recurrent neural network composed of LSTM layers to predict the actual cruise speed on each road segment. Since the LSTM layer has sequential dependency, we use a bidirectional LSTM layer to capture complete information and better utilize sequential sensitivity. Bi-LSTM uses two regular LSTM layers, each layer processes the input vector in a different direction (in time sequence and in reverse time sequence), and then combines their output vectors.

[0158] Although Bi-LSTM can capture long-term dependencies, each position in the input path is considered to have the same importance. In fact, some important road segments in the path may have a great impact on speed control, so more attention should be paid. Therefore, a self-attention mechanism is adopted, which can be represented as:

[0159]

[0160] e t,t′ = σ(W a α t,t′ +b a ),

[0161]

[0162]

[0163] where h t and h t are the input and output hidden vectors, W t , W x , b t and b a are the parameters or matrices to be learned. First, a mapping layer is adopted to convert the time series path into a sequence of vectors. Then, two bidirectional LSTM layers with loss and loop loss masks are used to extract the path features in chronological and reverse chronological order. In addition, an attention mechanism is adopted, so that more emphasis is placed on meaningful road segments. Finally, two fully connected layers with activation functions and loss layers are used to output the ratio of cruising speed to normal driving speed on each road segment. As shown in Figure 3 , the activation function of the final layer is “Softsign+1”, so the value range is p i ∈(0,2). By limiting the ratio of the desired cruising speed to the real-time normal driving speed within a reasonable range, the practicality of the cruising speed control scheme is enhanced.

[0164] Considering the dynamic nature of traffic conditions, the cruising speed of each road segment designed in this embodiment is time-varying. For the cruising path τ* = {s i , s j , s k ,...s z} formed by example one, the attention Bi-LSTM model outputs the ratio of cruising speed to normal driving speed on each road, denoted as {p i , p j , p k ,...p z}. When p i ≈1, it means that the taxi is driving at normal speed on road segment i. When p i <<1 or p i >>1, it means that the taxi driver intentionally reduces / increases the cruising speed on road segment i. In different time intervals, the desired cruising speed on road segment i is affected by p i and the current normal driving speed v ithe combined effects of the vehicle's speed and the vehicle's acceleration. The predicted path and speed of the vehicle are shown in Figure 4

[0165] Embodiment Three

[0166] A structure diagram of an automatic driving taxi cruise path planning device is provided in an embodiment of the present application. The embodiment of the present application provides services for implementation of the method of the above-mentioned embodiment one of the present application. Figure 3 A block diagram of an example device 12 suitable for implementing an embodiment of the present application is shown. Figure 3 The device 12 shown is merely one example and should not be taken as limiting the scope of the embodiments of the present application.

[0167] As shown in Figure 3 The device 12 is shown in the form of a general purpose computing device. The components of device 12 can include, but are not limited to, one or more processors or processing units 16, a system memory 28, and a bus 18 that couples various system components including system memory 28 to processor 16. Device 12 typically includes a variety of computer system readable media. Such media can be any available media that is accessible by device 12 and includes both volatile and non-volatile media, removable and non-removable media.

[0168] The system memory 28 can include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. Device 12 can further include other removable / non-removable, volatile / non-volatile computer system storage media. Program / utility 40 having a set (at least one) of program modules 42 can be stored in, for example, memory 28 by way of example, including an operating system, one or more application programs, other program modules, and program data, each of or some combination of which can provide functionality described herein. Program modules 42 generally carry out the functions and / or methodologies of embodiments of the present application described herein.

[0169] Device 12 can also communicate with one or more external devices 14 such as a keyboard, a pointing device, a display 24, etc.; one or more devices that enable a user to interact with device 12; and / or one or more devices that enable device 12 to communicate with one or more other computing devices. Such communication can be via input / output (I / O) interfaces 22. Further, device 12 can communicate with one or more networks such as a local area network (LAN), a general wide area network (WAN), and / or a public network such as the Internet, via network adapter 20. As Figure 3 ​As shown, network adapter 20 communicates with other modules of device 12 via bus 18. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0170] The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing the method provided in Embodiment 1 of the present invention.

[0171] Example 4

[0172] This embodiment provides a cruise speed planning device for an autonomous taxi, the structure of which is the same as that in Embodiment 3. Figure 3 Consistent, used to provide services for the implementation of the method of Embodiment 2 of the present invention. Figure 3 A block diagram of an exemplary device 12 suitable for implementing embodiments of the present invention is shown. Figure 3 The device 12 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0173] like Figure 3 As shown, device 12 is represented as a general-purpose computing device. Components of device 12 may include, but are not limited to: one or more processors or processing units 16, system memory 28, and a bus 18 connecting different system components, including system memory 28 and processing units 16. Device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by device 12, including volatile and non-volatile media, removable and non-removable media.

[0174] System memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. Device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. A program / utility 40 having a set (at least one) of program modules 42 may be stored in, for example, memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 42 typically perform the functions and / or methods described in the embodiments of the present invention.

[0175] Device 12 can also communicate with one or more external devices 14 such as a keyboard, a pointing device, a display 24, etc.; one or more devices that enable a user to interact with device 12; and / or any devices (e.g., network card, modem, etc.) that enable device 12 to communicate with one or more other computing devices. Such communication can occur via Input / Output (I / O) interface(s) 22. Still yet, device 12 can communicate with one or more networks, such as a local area network (LAN), a general wide area network (WAN), and / or a public network (e.g., the Internet) via network adapter 20. As depicted, network adapter 20 communicates with the other components of device 12 via bus 18. It should be appreciated that although not shown, other hardware and / or software modules could be used in conjunction with device 12. Such modules include, but are not limited to, microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archival storage systems, etc. ​

[0176] Processing unit 16 executes various functions applications and data processing by running programs stored in system memory 28, such as implementing the method provided by embodiment two of the present application.​

Claims

1. An automated taxi cruise path planning method, characterized by The method comprises the following steps: (1) obtaining historical cruise paths of a plurality of taxi experienced drivers, and dividing according to a preset rule to obtain a plurality of combinations comprising a training set and a test set; (2) learning the road network data using a preset path reward learning model to obtain a preliminary path reward; wherein the preset path reward learning model specifically comprises a one-hot encoding layer, a first full connection layer, a second full connection layer, a first normalization layer, a third full connection layer, a second normalization layer and a fourth full connection layer connected in sequence, and the preliminary path reward comprises a reward of each road segment in the road; (3) calculating a cruise strategy based on the preliminary path reward and the road network condition; (4) selecting a training set of any combination, and learning a first expected path access frequency of the current training set based on the cruise strategy; (5) updating the preset path reward learning model according to the first expected path access frequency and a corresponding first actual path access frequency; (6) judging whether the combination is traversed, if yes, executing step (7), otherwise returning to step (2); (7) for each preliminary planning reward, learning a second expected path access frequency of the corresponding test set based on the cruise strategy, and calculating a coefficient of the preliminary planning reward in combination with a corresponding second actual path access frequency; (8) linearly combining all preliminary planning rewards based on the coefficient to obtain a comprehensive planning reward; (9) determining an optimal cruise path according to the final path reward; Wherein, step (4) specifically comprises: (4-1) Selecting a training set T of any combination i from which the maximum path length U is obtained; (4-2) Retrieving the number of paths with length 1 starting from road segment s from the training set T i and assigning it to the predicted visit frequency initial value E 1,s,1 (s), s ∈ S, S is the set of all road segments in the road network; (4-3) calculating the expected access frequency according to the cruise strategy according to the following formula: E l+1,s,: (s) = ∑ S′∈s Σ a∈A P(s | s', a) π(a | s') E l,s′,: (s'), s e S, / = 1,..., U where E l+1,s,: (s) represents the predicted visit frequency of road segment s at the lth iteration, E l+1,s,: (s) = {E l+1,s,1 (s), …, E l+1,s,U (s)}, P(s | s ′ , a) represents the probability of entering road segment s from road segment s′ by performing action a, A represents the set of driving actions, and π(a | s ′ ) represents the probability of performing action a in the cruising strategy for road segment s ′ . (4-4) calculating the first expected path access frequency according to the expected access frequency according to the following formula: In the formula, E l,s,u (s) represents E l,s,: The u-th element in (s), μ exp [T i ] represents the first expected path access frequency.

2. The method of claim 1, wherein: The preset rule in step (1) is specifically: All historical cruise paths are taken as a basic data set Ω, a part is randomly selected from the basic data set Ω, and the selected cruise path is taken as a training set T of a current combination i , and the remaining cruise path is taken as a corresponding test set V i , a total of n times of selection is performed, n combinations are obtained, i represents a combination serial number, wherein T1≠T2...≠T i ...≠T n , and V1≠V2...V i ...≠V n .

3. The method of claim 1, wherein: Step (2) specifically comprises: (2-1) obtaining road network data, which is represented by a directed graph G=(V,E), V and E are respectively the set of road segments and intersections; (2-2) inputting the road network data into the preset path reward learning model to obtain a preliminary path reward, wherein the preset path reward learning model specifically comprises a one-hot encoding layer, a first full connection layer, a second full connection layer, a first normalization layer, a third full connection layer, a second normalization layer and a fourth full connection layer connected in sequence, and the preliminary path reward comprises a reward of each road segment in the road.

4. The method of claim 1, wherein: Step (3) specifically comprises: (3-1) generating a transition probability of each road segment to other road segments according to the road network condition; (3-2) setting the initial value V0(s)=-∞ of the value of all road segments in the road network, s∈S, S is the set of all road segments in the road network; (3-3) setting the iteration number t=1; (3-4) calculating the predicted reward of each road segment in the current iteration based on the preliminary path reward according to the following formula: Q t (s,a) = r i (s) + γ∑ s′∈s P(s' | s,a) · V t-1 (s'), s e S, a e A where Q t (s, a) denotes the predicted reward of performing action a on segment s, r i (s) denotes the reward of segment s in the preliminary path reward, γ denotes the discount factor, P(s ′ |s,a) denotes the probability of entering segment s' from segment s performing action a, A denotes the set of driving actions; (3-5) updating the value of each road segment according to the following formula: V t (s) = log∑ a∈A expQ t (s, a), s ∈ S V t (s) denotes the value of link s at the tth iteration; (3-6) judging whether t reaches a preset iteration number T, if yes, executing step (3-7), otherwise setting t=t+1 and returning to step (3-4); (3-7) According to the predicted reward value Q of each path segment at the end of iteration T (s, a) and the value V T (s), the cruise strategy is calculated: π(a|s) = exp(Q T (s,a)-V T (s)), s e S, a e A π={π(a|s)|s∈S,a∈A} In the formula, π represents a cruise strategy, and π (a|s) represents a probability of performing an action a on a road segment s.

5. The method of claim 1, wherein: Step (5) specifically comprises: (5-1) calculating a first actual path access frequency according to the training set; (5-2) calculating a regularization derivative of the preset path reward learning model according to the first expected path access frequency and the first actual path access frequency, according to the following formula: In the formula, Let φ denote the regularized derivative of the pre-defined path reward learning model, L denote the objective function of the pre-defined path reward learning model, and φ i μ represents the network parameters of the current preset path reward learning model. exp [T i ] represents the first expected path access frequency, μ real [T i [Indicates the frequency of access to the first actual path] Indicates the initial path reward with respect to φ i The derivative of L represents the regularization term. φ Indicates the target value for regularization; (5-3) updating network parameters of the preset path reward learning model according to the regularization derivative through back propagation.

6. The method of claim 1, wherein: Step (7) specifically comprises: (7-1) obtaining a corresponding test set for each preliminary planning reward; (7-2) obtaining a second expected path access frequency of each test set according to the cruise strategy learning; (7-3) calculating a second actual path access frequency of each test set; (7-4) updating a weight formula according to the second expected path access frequency and the second actual path access frequency, according to the following formula: wherein k i,j denotes the learning error rate, denote the second expected path visit frequency and the second actual path visit frequency of the link j in the test set respectively, m denotes the total number of links in the test set, w i,j , w i+1,j denote the weight of the link j in the preset path reward learning model corresponding to the i, i+1th preliminary planning reward respectively, η is a preset threshold, a i,j denotes the coefficient of the reward for the link j in the i th preliminary planning reward; (7-5) calculating a coefficient of each preliminary planning reward, according to the following formula: where a i represents the coefficient of the i-th preliminary planning reward.

7. An automated taxi cruise speed planning method characterized by comprising the following steps: (1) planning a path according to the method of any one of claims 1-6 to obtain an optimal planning path; (2) inputting the optimal planning path into a preset self-attention Bi-LSTM model to obtain a speed of each road segment in the optimal planning path, the preset self-attention Bi-LSTM model comprising a mapping layer, a Bi-LSTM network, a self-attention layer, a first full connection layer, a random inactivation layer, and a second full connection layer connected in sequence.

8. An autonomous taxi cruise path planning device characterized by comprising: comprising: one or more processors; a memory for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the method of any one of claims 1-6.

9. An autonomous taxi cruise speed planning apparatus characterized by comprising: comprising: one or more processors; a memory for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the method of claim 7.

Images