A method and system for generating intersection guide lines based on conditional variational autoencoder

Abstract

The present invention relates to a method and system for generating guide lines at intersections based on a conditional variational autoencoder, the method comprising: obtaining one or more non-guide line trajectories of a target intersection, encoding the sampling points in each of the non-guide line trajectories into a vector of fixed dimension using a neural network; extracting semantic features of the vector of fixed dimension; inputting the categories and semantic features of one or more of the non-guide line trajectories into a trained conditional variational autoencoder to obtain guide lines at the target intersection. The present invention mines implicit road traffic information from historical trajectory data based on a generative model in deep learning, and generates guide line trajectories that are more in line with road traffic rules and driving behaviors.

Description

Technical Field

[0001] The present invention belongs to the field of high-precision map production, and in particular relates to a method and system for generating intersection guide lines based on a conditional variational autoencoder. Background Art

[0002] In crowdsourcing high-precision map production, intersection guide lines are key elements in the map, which are used to guide smart driving cars to pass through intersections comfortably.

[0003] Traditional methods for generating intersection guide lines are mostly based on kinematic models, and generate guide lines by solving a multi-objective non-convex optimization problem. This method assumes that road traffic elements such as POIs in the map are ideal and complete, proposes objective functions in advance, and designs optimization rules and constraints such as collision avoidance and speed. Although it can generate smooth guide lines through mathematical methods, it does not take into account the actual road conditions and traffic information, resulting in the dilemma that guide lines are difficult to apply in practice.

[0004] There is uncertainty in the perception of road elements in crowdsourced high-precision maps, and the road elements in the cloud composition are not necessarily complete, which further exacerbates the problem that the traditional guide lines generated based on kinematic models are difficult to use. With the development of deep learning, there have been a lot of studies on using a large amount of trajectory data in crowdsourced data for trajectory planning and trajectory generation. Compared with traditional methods, the deep learning-based method is a data-driven method that takes into account actual road conditions and relies less on expert experience. It is a promising end-to-end trajectory planning and generation method. Summary of the invention

[0005] In order to solve the problem that traditional trajectory generation methods rely on expert experience and are difficult to apply to actual road conditions, a first aspect of the present invention provides a method for generating intersection guide lines based on a conditional variational autoencoder, including: obtaining one or more non-guide line trajectories of a target intersection, using a neural network to encode the sampling points in each of the non-guide line trajectories into a vector of fixed dimension; extracting semantic features of the vector of fixed dimension; according to the category and semantic features of one or more of the non-guide line trajectories, inputting them into a trained conditional variational autoencoder to obtain the guide line of the target intersection.

[0006] In some embodiments of the present invention, encoding each sampling point in the non-guide line trajectory into a vector of fixed dimension by using a neural network includes: implementing feature crossover and encoding into a vector of fixed dimension by a fully connected neural network.

[0007] In some embodiments of the present invention, the conditional variational autoencoder includes a convolutional neural network and an LSTM, wherein the convolutional neural network is used to output a style vector based on the category and semantic features of one or more of the non-guide line trajectories; and to generate trajectory points based on the predicted trajectory point sequence output by the LSTM; and the LSTM is used to generate a predicted trajectory point sequence based on the style vector, the start point and the end point of the non-guide line trajectory.

[0008] Furthermore, the convolutional neural network includes a first convolutional neural network and a second convolutional neural network, the first convolutional neural network is used to output a style vector based on the category and semantic features of one or more of the non-guide line trajectories; the second convolutional neural network is used to generate trajectory points based on the predicted trajectory point sequence output by LSTM.

[0009] Furthermore, outputting a style vector according to the category and semantic features of one or more of the non-guide line trajectories includes: fusing the category and semantic features of one or more of the non-guide line trajectories through a fully connected layer, and outputting the logarithm of the mean and square of the variance of the prior distribution; obtaining a sampling vector from the prior distribution; and outputting a style vector according to the sampling vector and the category of the non-guide line trajectory.

[0010] Preferably, the second convolutional neural network is trained by the following method: obtaining the predicted trajectory point sequence output by LSTM and the guide line trajectory of the corresponding intersection; using the predicted trajectory point sequence as a sample and the guide line trajectory of the corresponding intersection as a label to construct a training set; using the training set to train the second convolutional neural network until its loss function value is lower than a threshold and tends to be stable, thereby obtaining a trained second convolutional neural network.

[0011] The second aspect of the present invention provides a system for generating intersection guide lines based on a conditional variational autoencoder, comprising: an acquisition module for acquiring one or more non-guide line trajectories of a target intersection, and using a neural network to encode the sampling points in each of the non-guide line trajectories into a vector of fixed dimension; an extraction module for extracting semantic features of the vector of fixed dimension; and a generation module for inputting the categories and semantic features of one or more of the non-guide line trajectories into a trained conditional variational autoencoder to obtain the guide lines of the target intersection.

[0012] Furthermore, the generation module includes a conditional variational autoencoder including a convolutional neural network and an LSTM, wherein the convolutional neural network is used to output a style vector according to the category and semantic features of one or more of the non-guide line trajectories; and to generate trajectory points according to the predicted trajectory point sequence output by the LSTM; and the LSTM is used to generate a predicted trajectory point sequence according to the style vector, the starting point and the end point of the non-guide line trajectory.

[0013] The third aspect of the present invention provides an electronic device, comprising: one or more processors; a storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the intersection guide line generation method based on conditional variational autoencoder provided in the first aspect of the present invention.

[0014] A fourth aspect of the present invention provides a computer-readable medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the method for generating intersection guide lines based on a conditional variational autoencoder provided in the first aspect of the present invention is implemented.

[0015] The beneficial effects of the present invention are:

[0016] 1. Based on the generative model in deep learning, the present invention mines implicit road traffic information from historical trajectory data in crowdsourcing and generates a guide line trajectory that is more in line with road traffic rules and driving behavior;

[0017] 2. The present invention realizes that the historical trajectory data can be obtained from the collection end (such as the collection vehicle), that is, a data-driven trajectory generation model is realized from the collection end to the decision end (such as the automatic driving end). BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A basic flow chart of a method for generating a guide line at an intersection based on a conditional variational autoencoder in some embodiments of the present invention;

[0019] Figure 2 A schematic diagram of a specific flow chart of a method for generating a guide line at an intersection based on a conditional variational autoencoder in some embodiments of the present invention;

[0020] Figure 3 A schematic diagram of the effect of intersection guide lines generated based on a conditional variational autoencoder in some embodiments of the present invention;

[0021] Figure 4 Schematic diagram of the specific structure of a system for generating guide lines at intersections based on a conditional variational autoencoder in some embodiments of the present invention;

[0022] Figure 5 It is a schematic diagram of the structure of an electronic device in some embodiments of the present invention. DETAILED DESCRIPTION

[0023] The principles and features of the present invention are described below in conjunction with the accompanying drawings. The examples given are only used to explain the present invention and are not used to limit the scope of the present invention.

[0024] refer to Figure 1In a first aspect of the present invention, a method for generating intersection guide lines based on a conditional variational autoencoder is provided, comprising: S100. obtaining one or more non-guide line trajectories of a target intersection, and using a neural network to encode the sampling points in each of the non-guide line trajectories into a vector of fixed dimension; S200. extracting semantic features of the vector of fixed dimension; S300. inputting the categories and semantic features of one or more of the non-guide line trajectories into a trained conditional variational autoencoder (CVAE) to obtain the guide lines of the target intersection.

[0025] In S100 of some embodiments of the present invention, in order to facilitate the processing of the subsequent model, the non-guide line trajectory or the guide line trajectory needs to be encoded, that is, the original trajectory data is encoded into a vector of fixed dimension. The use of a neural network to encode each sampling point in the non-guide line trajectory into a vector of fixed dimension includes: implementing feature crossover and encoding into a vector of fixed dimension through a fully connected neural network.

[0026] Specifically, the following steps are included: S101. Local coordinate transformation of sampling points in the trajectory; mainly through three local coordinate transformation methods: Cartesian coordinates in the same direction, Cartesian coordinates in the lane direction, and trajectory behavior sequence. The specific coordinate transformation method will not be repeated here; S102. Encoding of sampling points in the trajectory; the features of the sampling points in the trajectory are mainly longitude and latitude, and the features are crossed and encoded into vectors of fixed dimensions through a fully connected neural network.

[0027] To facilitate feature extraction and generation of the conditional variational autoencoder, in S300 of some embodiments of the present invention, the conditional variational autoencoder includes a convolutional neural network and an LSTM, wherein the convolutional neural network is used to output a style vector based on the category and semantic features of one or more of the non-guide line trajectories; and to generate trajectory points based on the predicted trajectory point sequence output by the LSTM; and the LSTM is used to generate a predicted trajectory point sequence based on the style vector, the start point and the end point of the non-guide line trajectory.

[0028] It can be understood that LSTM has an advantage in extracting time series features, while convolutional neural networks have an advantage in processing multi-dimensional data (multi-dimensional vectors). Combining the two can effectively speed up the convergence process of the model and thus improve the rationality of the generated trajectory point sequence.

[0029] Furthermore, the convolutional neural network includes a first convolutional neural network and a second convolutional neural network, wherein the first convolutional neural network is used to output a style vector according to the category and semantic features of one or more non-guide line trajectories; and the second convolutional neural network is used to generate trajectory points according to the predicted trajectory point sequence output by the LSTM. It can be understood that the style vector can be regarded as a classification of different trajectories formed due to different driving styles. For example, the safety type style prefers stability, and its guide line trajectory is gentle and has little change; the strategic type selects the most suitable route according to different road conditions and traffic conditions, and its guide line trajectory changes unpredictably.

[0030] Furthermore, the outputting of the style vector according to the category and semantic features of one or more non-guide line trajectories includes: fusing the category and semantic features of one or more non-guide line trajectories through a fully connected layer, and outputting the logarithm of the mean and square of the variance of the prior distribution; obtaining a sampling vector from the prior distribution; and outputting the style vector according to the sampling vector and the category of the non-guide line trajectories. Specifically, the semantic features of the trajectories and the type of trajectories are combined to predict the mean and variance of the prior distribution through a multi-layer fully connected neural network.

[0031] Preferably, the second convolutional neural network is trained by the following method: obtaining the predicted trajectory point sequence output by LSTM and the guide line trajectory of the corresponding intersection; using the predicted trajectory point sequence as a sample and the guide line trajectory of the corresponding intersection as a label to construct a training set; using the training set to train the second convolutional neural network until its loss function value is lower than a threshold and tends to be stable, thereby obtaining a trained second convolutional neural network.

[0032] Optionally, the loss function is:

[0033]

[0034]

[0035] Among them, loss represents the loss function, KL[.] represents the KL divergence of the expression in the brackets, L[.] represents the Lagrangian function, and E[.] represents the expectation of the expression in the brackets; X represents the distribution of the set of guide line trajectories generated by the second convolutional neural network, Z represents the distribution of the set of actual guide line trajectories, θ, They represent the generated guide line trajectory and the actual guide line trajectory respectively; p θ is the probability of the generated guide line trajectory occurring, is the probability of the actual guide line trajectory occurring. The training algorithm is the BP algorithm based on mini-batch, and the learning strategy is RMSprop.

[0036] In one possible embodiment, schematically, Figure 2 The trajectory generation network structure based on CVAE is shown. i ,y i They represent the values ​​of the local coordinates of the trajectory sampling points after transformation, Emb represents the fully connected neural network, and e i is the encoded feature vector, i corresponds to the sampling point in the trajectory. LSTM is a recurrent neural network that extracts the spatiotemporal features of the trajectory sequence. h is the output of LSTM. FCi represents the fully connected neural network i. Type is the trajectory type, and c is the condition of CVAE. μ,logσ 2 They represent the logarithmic values ​​of the mean and square of the prior distribution predicted by the network. z is the sampling vector of the prior distribution. Style represents the trajectory style vector after sampling z and condition c. End point and [0, 0] represent the local end point and starting point of the known trajectory points to be generated.

[0037] Indicatively, Figure 3 The schematic diagram of the effect of the intersection guide line generated by the intersection guide line generation method based on conditional variational autoencoder of the present invention is shown; the largest octagonal box is the road-level intersection range, the lane line is outside the octagonal box, and the line without small triangle marks in the box is a virtual lane line with no practical meaning. The line with small triangle marks is the guide line generated based on the trajectory representation method of the present invention. It can be seen from the figure that it has smooth lines, natural transitions, and the trajectory is consistent with normal driving behavior.

[0038] Example 2

[0039] refer to Figure 4 In a second aspect of the present invention, a system 1 for generating guide lines for intersections based on a conditional variational autoencoder is provided, comprising: an acquisition module 11 for acquiring one or more non-guide line trajectories of a target intersection, and encoding the sampling points in each of the non-guide line trajectories into a vector of a fixed dimension using a neural network; an extraction module 12 for extracting semantic features of the vector of the fixed dimension; and a generation module 13 for inputting the categories and semantic features of one or more of the non-guide line trajectories into a trained conditional variational autoencoder to obtain guide lines for the target intersection.

[0040] Furthermore, the generation module 13 includes a conditional variational autoencoder including a convolutional neural network and an LSTM, wherein the convolutional neural network is used to output a style vector according to the category and semantic features of one or more of the non-guide line trajectories; and to generate trajectory points according to the predicted trajectory point sequence output by the LSTM; and the LSTM is used to generate a predicted trajectory point sequence according to the style vector, the starting point and the end point of the non-guide line trajectory.

[0041] Example 3

[0042] refer to Figure 5 According to the third aspect of the present invention, there is provided an electronic device, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method of the first aspect of the present invention.

[0043] The electronic device 500 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage device 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the electronic device 500 are also stored. The processing device 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0044] Typically, the following devices may be connected to the I / O interface 505: an input device 506 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 507 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 508 including, for example, a hard disk, etc.; and a communication device 509. The communication device 509 may allow the electronic device 500 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 5 The electronic device 500 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 5 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0045] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through a communication device 509, or installed from a storage device 508, or installed from a ROM 502. When the computer program is executed by the processing device 501, the above functions defined in the method of the embodiment of the present disclosure are executed. It should be noted that the computer-readable medium described in the embodiment of the present disclosure can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection with one or more conductors, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In an embodiment of the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In an embodiment of the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to: wire, optical cable, RF (radio frequency), etc., or any suitable combination of the foregoing.

[0046] The computer-readable medium may be included in the electronic device, or may exist independently without being installed in the electronic device. The computer-readable medium carries one or more computer programs. When the one or more programs are executed by the electronic device, the electronic device:

[0047] Computer program code for performing the operations of embodiments of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, Python, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on a user's computer, partially on a user's computer, as a separate software package, partially on a user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0048] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0049] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for generating intersection guide lines based on conditional variational autoencoder, characterized in that: include: Obtain one or more non-guideline trajectories of the target intersection, and use a neural network to encode sampling points in each of the non-guideline trajectories into a vector of fixed dimension; Extracting semantic features of the vector of fixed dimension; According to the category and semantic features of one or more non-guide line trajectories, the non-guide line trajectories are input into a trained conditional variational autoencoder to obtain a guide line of the target intersection; The conditional variational autoencoder includes a convolutional neural network and an LSTM, wherein the convolutional neural network is used to output a style vector according to the category and semantic features of one or more non-guide line trajectories; and to generate trajectory points according to the predicted trajectory point sequence output by the LSTM; wherein the convolutional neural network includes a first convolutional neural network and a second convolutional neural network, wherein the first convolutional neural network is used to output a style vector according to the category and semantic features of one or more non-guide line trajectories; the outputting of the style vector according to the category and semantic features of one or more non-guide line trajectories includes: fusing the category and semantic features of one or more non-guide line trajectories through a fully connected layer, and outputting the logarithm of the mean and square of the variance of the prior distribution; obtaining a sampling vector from the prior distribution; outputting a style vector according to the sampling vector and the category of the non-guide line trajectory; the style vector is a classification of different trajectories formed due to different driving styles; The second convolutional neural network is used to generate trajectory points according to the predicted trajectory point sequence output by the LSTM; the LSTM is used to generate the predicted trajectory point sequence according to the style vector and the starting point and end point of the non-guide line trajectory.

2. The method for generating intersection guide lines based on conditional variational autoencoder according to claim 1, characterized in that: The step of encoding each sampling point in the non-guide line trajectory into a vector of fixed dimension by using a neural network comprises: Feature cross-talk and encoding into a vector of fixed dimension are achieved through a fully connected neural network.

3. The method for generating intersection guide lines based on conditional variational autoencoder according to claim 1, characterized in that: The second convolutional neural network is trained by the following method: Get the predicted trajectory point sequence output by LSTM and the guide line trajectory of the corresponding intersection; The predicted trajectory point sequence is used as a sample and the guide line trajectory of the corresponding intersection is used as a label to construct a training set; The second convolutional neural network is trained using the training set until its loss function value is lower than a threshold and tends to be stable, thereby obtaining a trained second convolutional neural network.

4. A system for generating intersection guide lines based on conditional variational autoencoder, characterized in that: include: An acquisition module, used to acquire one or more non-guide line trajectories of a target intersection, and encode the sampling points in each of the non-guide line trajectories into a vector of fixed dimension using a neural network; An extraction module, used for extracting semantic features of the vector of fixed dimension; A generation module, configured to input the categories and semantic features of one or more non-guideline trajectories into a trained conditional variational autoencoder to obtain a guideline of a target intersection; The conditional variational autoencoder includes a convolutional neural network and an LSTM, wherein the convolutional neural network is used to output a style vector according to the category and semantic features of one or more non-guide line trajectories; and to generate trajectory points according to the predicted trajectory point sequence output by the LSTM; wherein the convolutional neural network includes a first convolutional neural network and a second convolutional neural network, wherein the first convolutional neural network is used to output a style vector according to the category and semantic features of one or more non-guide line trajectories; the outputting of the style vector according to the category and semantic features of one or more non-guide line trajectories includes: fusing the category and semantic features of one or more non-guide line trajectories through a fully connected layer, and outputting the logarithm of the mean and square of the variance of the prior distribution; obtaining a sampling vector from the prior distribution; outputting a style vector according to the sampling vector and the category of the non-guide line trajectory; the style vector is a classification of different trajectories formed due to different driving styles; the second convolutional neural network is used to generate trajectory points according to the predicted trajectory point sequence output by the LSTM; the LSTM is used to generate a predicted trajectory point sequence according to the style vector, the starting point and the end point of the non-guide line trajectory.

5. An electronic device comprising: one or more processors; A storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, so that the one or more processors implement the intersection guide line generation method based on conditional variational autoencoder as described in any one of claims 1 to 3.

6. A computer readable medium having a computer program stored thereon, wherein: When the computer program is executed by a processor, the method for generating intersection guide lines based on a conditional variational autoencoder according to any one of claims 1 to 3 is implemented.

Images