Action marking method and device, electronic equipment and storage medium

Abstract

The invention discloses an action marking method and device, electronic equipment and a storage medium, and the method comprises the steps: predicting a transverse token according to the curvature and vehicle speed of a vehicle at a current moment; predicting a longitudinal token according to the transverse token and the vehicle speed and the acceleration of the vehicle at the current moment; and in response to the driving track of the transverse token and the driving track of the longitudinal token, updating the driving track of the vehicle at the next moment. According to the method, the trajectory generation result is obtained through the transverse token prediction and the longitudinal token. And meanwhile, the number of the tokens can be reduced, and the calculation amount of the tokens is reduced.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, and in particular to a method, apparatus, electronic device, and storage medium for action tagging. Background Technology

[0002] The core of autonomous driving lies in the real-time processing and decision-making of massive amounts of multimodal sensor data. In the domain of large models, tokens represent the smallest units of information after preprocessing. Relevant data for autonomous driving can be transformed into discrete symbols or vector representations through tokenization.

[0003] Tokenization, a crucial step in data preprocessing, aims to transform raw data into discrete, easily processed basic units. Through appropriate tokenization strategies, models can reduce redundant information, ensure the representation of key information, shorten input sequence length, and significantly improve training and inference efficiency.

[0004] In related technologies, the clustering results of autonomous driving information after tokenization are not ideal, as they are affected by noise and lack interdependencies between data. Summary of the Invention

[0005] This application provides an action tokenization method, device, electronic device, and storage medium to achieve tokenization through horizontal and vertical linkage and generate a planned path.

[0006] The embodiments of this application adopt the following technical solutions:

[0007] In a first aspect, embodiments of this application provide an action tokenization method, wherein the method includes:

[0008] Predict the lateral token based on the vehicle's curvature and speed at the current moment;

[0009] Based on the lateral token, the vehicle's current speed and acceleration, predict the longitudinal token;

[0010] In response to the driving trajectory of the lateral token and the driving trajectory of the longitudinal token, the vehicle's driving trajectory at the next moment is updated.

[0011] In some embodiments, predicting the lateral token based on the vehicle's curvature and speed at the current moment includes:

[0012] The curvature and speed of the vehicle at the current moment are used as inputs to the lateral planning problem of the vehicle. The rate of change of curvature of the predicted path, the length of the path, and whether the vehicle is reversing are used as discrete prediction variables. The prediction results of the lateral token are obtained by tokenizing the output.

[0013] In some embodiments, predicting the lateral token based on the vehicle's curvature and speed at the current moment further includes:

[0014] Based on the prediction results of the lateral token and the spiral formula, the lateral token is converted into a path as the trajectory of the lateral token.

[0015] In some embodiments, predicting the longitudinal token based on the lateral token, the vehicle's current speed, and acceleration includes:

[0016] The vehicle's current speed, acceleration, and the lateral token are used as inputs to the vehicle's longitudinal planning problem. The target speed, target acceleration, and time duration of the vehicle are predicted, and the result is tokenized to obtain the prediction result of the longitudinal token.

[0017] In some embodiments, predicting the longitudinal token based on the lateral token, the vehicle's current speed, and acceleration further includes:

[0018] Based on the prediction result of the vertical token, the vertical token is converted into a path as the trajectory of the vertical token.

[0019] In some embodiments, the method further includes:

[0020] Establish the correlation between the horizontal token and the vertical token, such that the labeling result of the horizontal token at each time step affects the prediction result of the vertical token; and / or the labeling result of the vertical token at each time step inversely affects the prediction result of the horizontal token.

[0021] In some embodiments, updating the vehicle's trajectory at the next moment in response to the lateral token's trajectory and the longitudinal token's trajectory includes:

[0022] The vehicle's path is updated based on the lateral token's travel trajectory;

[0023] Update the vehicle's trajectory based on the longitudinal token and the path.

[0024] Secondly, embodiments of this application also provide an action tagging device, wherein the device includes:

[0025] The lateral token prediction module is used to predict the lateral token based on the vehicle's curvature and speed at the current moment.

[0026] The longitudinal token prediction module is used to predict the longitudinal token based on the lateral token, the vehicle's speed and acceleration at the current moment;

[0027] The trajectory generation module is used to update the vehicle's trajectory at the next moment in response to the driving trajectory of the lateral token and the driving trajectory of the longitudinal token.

[0028] Thirdly, embodiments of this application also provide an electronic device, including: a processor; and a memory arranged to store computer-executable instructions, which, when executed, cause the processor to perform the above-described method.

[0029] Fourthly, embodiments of this application also provide a computer-readable storage medium that stores one or more programs, which, when executed by an electronic device including multiple applications, cause the electronic device to perform the above-described method.

[0030] The at least one technical solution adopted in this application embodiment can achieve the following beneficial effects: First, a lateral token is predicted based on the vehicle's curvature and speed at the current moment; then, a longitudinal token is predicted based on the lateral token, the vehicle's speed and acceleration at the current moment; and the vehicle's trajectory at the next moment is updated by responding to the lateral token's trajectory and the longitudinal token's trajectory. Through the above method, by separately performing lateral and longitudinal token calculations, the final obtained trajectory can be used for path planning of the vehicle at the next moment, which not only satisfies the constraints of the vehicle dynamics model but is also easy to understand. Attached Figure Description

[0031] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0032] Figure 1 This is a schematic diagram illustrating the tokenization of the action tokenization method in the embodiments of this application;

[0033] Figure 2 This is a flowchart illustrating the action tagging method in the embodiments of this application;

[0034] Figure 3 This is a schematic diagram of the action marking device in the embodiments of this application;

[0035] Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0036] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0037] In the trajectory generation task of autonomous driving, action tokens (also known as "action tokenization") are a key step in converting driving behaviors (such as uniform acceleration, braking, steering, etc.) into a format that the generative model can understand. These representations of actions are one of the core pieces of information for autonomous driving systems when making decisions.

[0038] Specifically, Action tokens are a term used in the field of autonomous driving to encode vehicle behavior and the environment. They refer to the conversion of driving actions and scene elements into discretized data units that can be recognized by a model through specific encoding methods. By encoding actions such as acceleration, deceleration, and steering in driving decisions, as well as environmental elements (such as lane lines and obstacle positions), Action tokens help autonomous driving systems achieve end-to-end decision-making. For example, in a certain automaker's MindVLA architecture, Action tokens are combined with a 3D spatial encoder and a language model to generate real-time driving trajectory optimization schemes.

[0039] Research suggests that action tokens typically offer two options. One is to use a pre-trained tokenizer (such as VQ-VAE) to encode continuous actions as discrete tokens. However, this method relies on a large amount of raw data to train the tokenizer, and if the data volume is insufficient, the pre-trained tokenizer may not be able to effectively handle new data. The other approach is to distinguish each action as two scalar values ​​(e.g., velocity and curvature) and map these two scalar values ​​to a d-dimensional space through a linear layer, thus obtaining a d-dimensional representation. To ensure that the model can still generalize effectively with a small amount of data samples, methods based on explicit discretization have also emerged. These methods map continuous features to discrete tokens through simple normalization and equal-interval partitioning. This approach is intuitive and easy to implement, such as a simple discretization method combining time-time partitioning with k-disks clustering.

[0040] The solutions in related technologies still have shortcomings and deficiencies:

[0041] (1) Although linear mapping can provide some flexibility, this method may still be limited by the linear assumption. For some complex nonlinear control tasks, a simple linear layer may not be enough to capture the rich features of the control action.

[0042] (2) By treating velocity and curvature as independent scalars, the possible interrelationships between them are ignored. In some cases, changes in velocity and curvature are often interdependent, and simple linear mapping may not capture this interdependence.

[0043] (3) The k-disks clustering algorithm (a method for tokenizing trajectory data) is based on distance metrics. It can effectively handle high-dimensional features or complex spatial patterns in trajectory data and is easily scalable to large-scale datasets. However, the fixed time interval division method may lead to information loss or inaccurate representation. When stitching trajectories between different time periods, it may cause abrupt changes in curvature. In addition, the clustering results may be very sensitive to noise or outliers in the trajectory data. If there are errors, sensor noise, or atypical behavior in the trajectory, it may lead to inaccurate clustering. In addition, how to choose an appropriate k value (the specific number of clusters) is also a challenge. If the k value is set inappropriately, it may lead to clustering that is too coarse or too fine, affecting subsequent trajectory generation.

[0044] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0045] like Figure 1 As shown, unlike traditional methods that tokenize the entire trajectory (such as k-disks) or encode curvature and speed separately, the method in this embodiment takes the vehicle's current curvature and speed as inputs at time t. It treats the prediction of the next token based on curvature and speed as a joint lateral and longitudinal spatiotemporal planning problem. First, lateral actions are tokenized based on a spiral path. Then, longitudinal tokenization is performed. During longitudinal tokenization, whether the path is reversed, the rate of change of path curvature, and the path length are used for tokenization, ensuring that paths covering the entire scenario (including but not limited to parking and U-turns) can be generated. Furthermore, since the rate of change of curvature is predicted, the smoothness of path splicing between tokens is guaranteed. Finally, a joint trajectory is generated. Combining the lateral path and the longitudinal trajectory yields the vehicle's driving trajectory for the next time period. Optionally, to ensure the smoothness of the longitudinal trajectory, a fifth-order polynomial can be used to tokenize the longitudinal path.

[0046] This application provides an action tokenization method, such as... Figure 2 The diagram shows a flowchart of an action tokenization method in an embodiment of this application. The method includes at least the following steps S210 to S230:

[0047] Step S210: Predict the lateral token based on the curvature and speed of the vehicle at the current moment.

[0048] When generating trajectories for autonomous driving, the lateral token is first predicted based on the vehicle's path curvature and speed at the current time (time t). This means that during path planning, the path planning result can be generated based on factors such as vehicle speed and the rate of change of curvature.

[0049] When predicting lateral tokens, which corresponds to a lateral programming problem, there are at least three variables that need to be predicted: whether the token is reversed, the rate of change of curvature, and the path length.

[0050] It is important to note that whether it is reversed, i.e. whether it is traveling in the opposite direction of the current token's destination, this discretization helps in generating complex trajectories, such as parking scenarios or vehicle U-turn scenarios.

[0051] Step S220: Based on the lateral token, the vehicle's speed and acceleration at the current moment, predict the longitudinal token.

[0052] Based on the predicted lateral token, the vehicle speed and acceleration at the current time (time t), the longitudinal token is predicted.

[0053] Since the horizontal token calculation also includes the prediction of the rate of curvature change, ensuring the smoothness of the path splicing between tokens, the current speed, current acceleration, and path length of the vehicle are selected as query variables. The variables for predicting the vertical tokens include, but are not limited to, the target speed, the target acceleration, and the time length.

[0054] The input portion of the above-mentioned vertical token prediction includes the results of the above-mentioned horizontal token prediction, thus reflecting the correlation between vertical token prediction and horizontal token prediction.

[0055] Step S230: In response to the driving trajectory of the lateral token and the driving trajectory of the longitudinal token, update the driving trajectory of the vehicle at the next moment.

[0056] The driving trajectory of the vehicle at the next moment is determined based on the driving trajectories of the lateral token and the longitudinal token, respectively. It is understood that various methods from related technologies can be used to generate the driving trajectory using the lateral token and the longitudinal token, and this application does not limit the specific methods used. For example, the Frenet coordinate system can be used when generating the longitudinal trajectory, and the spiral curve formula can be used when generating the lateral trajectory.

[0057] The above method achieves lateral motion tokenization. Based on the curvature and speed of the vehicle at the current moment, a lateral token is predicted on the path. Then, based on the lateral token, the vehicle's speed and acceleration at the current moment, a longitudinal token is predicted in terms of speed.

[0058] Using the methods described above, joint horizontal and vertical action labeling can better coordinate and optimize horizontal and vertical token prediction. Horizontal and vertical trajectory planning can not only be performed independently but also influence each other, ultimately generating a more natural and reasonable integrated trajectory.

[0059] In one embodiment of this application, the step of predicting the lateral token based on the curvature and speed of the vehicle at the current moment includes: taking the curvature and speed of the vehicle at the current moment as input to the lateral planning problem of the vehicle, taking the rate of change of curvature of the predicted path, the length of the path, and whether the vehicle is reversing as discrete prediction variables, and tokenizing the output to obtain the prediction result of the lateral token.

[0060] Specifically, lateral token prediction corresponds to the lateral planning problem of the vehicle, mainly including three variables that need to be predicted: whether it is reversing, the rate of curvature change, and the path length. It is important to note that whether it is reversing refers to whether the vehicle is traveling in the opposite direction to the destination of the current token. This discretization is helpful for generating complex trajectories, such as parking scenarios or U-turn scenarios.

[0061] By including vehicle speed information in the predicted input, this reflects that when a vehicle travels along a curvature path at a certain speed, the selectable rate of curvature change and travel distance in the next moment are limited.

[0062] In one embodiment of this application, predicting the lateral token based on the curvature and speed of the vehicle at the current moment further includes: calculating, based on the prediction result of the lateral token and the spiral curve formula, converting the lateral token into a path as the driving trajectory of the lateral token.

[0063] After predicting the horizontal token, the horizontal path is generated. With the current vehicle position as the origin of the coordinate system (x0 = 0, y0 = 0, t0 = 0), the horizontal token can be converted into a path according to the predicted horizontal token and the spiral formula, so as to better interact with the current environment.

[0064] It is understood that the formula for the spiral curve is well known to those skilled in the art, and therefore will not be elaborated upon further.

[0065] If lateral motion is labeled based on a spiral trajectory, only three key variables need to be predicted. Specifically, when modeling the longitudinal trajectory using a fifth-order polynomial, only six key variables need to be calculated during trajectory generation. This simplifies the computation process and reduces reliance on large datasets.

[0066] In one embodiment of this application, the step of predicting the longitudinal token based on the lateral token, the vehicle's speed and acceleration at the current moment includes: taking the vehicle's speed and acceleration at the current moment and the lateral token as input to the vehicle's longitudinal planning problem, predicting the vehicle's target speed, target acceleration and time length, and tokenizing the output to obtain the prediction result of the longitudinal token.

[0067] Specifically, when predicting the longitudinal token, the vehicle's current speed, acceleration, and the lateral token are used as inputs to the vehicle's longitudinal planning problem, thereby predicting the vehicle's target speed, target acceleration, and time duration.

[0068] By tokenizing paths based on whether they are reversed, the rate of change of path curvature, and path length, it is ensured that paths covering all scenarios (such as reversing, parking, and U-turns) can be generated. Furthermore, since the rate of change of path curvature is predicted, the smoothness of path splicing between horizontal and vertical tokens is guaranteed. To ensure the smoothness of the vertical trajectory, methods known in related technologies can be used for tokenization of the vertical path; no specific limitations are imposed in this embodiment.

[0069] In one embodiment of this application, predicting the longitudinal token based on the lateral token, the vehicle's speed and acceleration at the current moment further includes: converting the longitudinal token into a path as the driving trajectory of the longitudinal token based on the prediction result of the longitudinal token.

[0070] For vertical token prediction, the current speed, current acceleration, and path length of the vehicle are selected as query variables, and the target speed, target acceleration, and time length are selected as prediction variables.

[0071] Vertical token prediction is followed by vertical trajectory generation.

[0072] In one embodiment of this application, the method further includes: establishing the correlation between the horizontal token and the vertical token, such that the labeling result of the horizontal token at each time affects the prediction result in the vertical token; and / or the labeling result of the vertical token at each time affects the prediction result in the horizontal token in turn.

[0073] The input to the prediction includes the results of the lateral token prediction, reflecting the correlation between the longitudinal and lateral token predictions. For example, if the lateral token reversal flag is true at the next time step, it indicates that the expected velocity of the longitudinal token at the end of the lateral path is 0. By using the reversal flag, rate of curvature change, and path length as discrete predictor variables, the model can handle more complex trajectory scenarios such as parking, U-turns, and reverse driving. Furthermore, the introduction of the reversal flag is particularly helpful in generating reasonable paths when vehicles need to reverse.

[0074] In other words, when the token's reverse flag is true at the next horizontal moment, it indicates that the path is reversed, and the corresponding vertical token has no speed at the end of the horizontal path, i.e., the speed is 0. There are many similar examples, but specific examples are not given in this application's embodiments.

[0075] In one embodiment of this application, updating the vehicle's trajectory at the next moment in response to the lateral token's trajectory and the longitudinal token's trajectory includes: updating the vehicle's path based on the lateral token's trajectory; and updating the vehicle's trajectory based on the longitudinal token's trajectory.

[0076] Using joint lateral and longitudinal action labeling can better coordinate and optimize lateral and longitudinal token prediction, updating the vehicle's path and trajectory. Lateral and longitudinal trajectory planning can not only be performed independently but also influence each other, ultimately generating a more natural and reasonable comprehensive trajectory, which is then used to update the vehicle's speed and / or acceleration at the next moment.

[0077] This application embodiment also provides an action tagging device 300, such as Figure 3 As shown, a schematic diagram of the action tokenization device in this embodiment is provided. The action tokenization device 300 includes at least: a horizontal token prediction module 310, a vertical token prediction module 320, and a trajectory generation module 330, wherein:

[0078] In one embodiment of this application, the lateral token prediction module 310 is specifically used to: predict the lateral token based on the curvature and speed of the vehicle at the current moment.

[0079] When generating trajectories for autonomous driving, the lateral token is first predicted based on the vehicle's path curvature and speed at the current time (time t). This means that during path planning, the path planning result can be generated based on factors such as vehicle speed and the rate of change of curvature.

[0080] When predicting lateral tokens, which corresponds to a lateral programming problem, there are at least three variables that need to be predicted: whether the token is reversed, the rate of change of curvature, and the path length.

[0081] It is important to note that whether it is reversed, i.e. whether it is traveling in the opposite direction of the current token's destination, this discretization helps in generating complex trajectories, such as parking scenarios or vehicle U-turn scenarios.

[0082] In one embodiment of this application, the longitudinal token prediction module 320 is specifically used to: predict the longitudinal token based on the lateral token, the vehicle's speed and acceleration at the current moment.

[0083] Based on the predicted lateral token, the vehicle speed and acceleration at the current time (time t), the longitudinal token is predicted.

[0084] Since the horizontal token calculation also includes the prediction of the rate of curvature change, ensuring the smoothness of the path splicing between tokens, the current speed, current acceleration, and path length of the vehicle are selected as query variables. The variables for predicting the vertical tokens include, but are not limited to, the target speed, the target acceleration, and the time length.

[0085] The input portion of the above-mentioned vertical token prediction includes the results of the above-mentioned horizontal token prediction, thus reflecting the correlation between vertical token prediction and horizontal token prediction.

[0086] In one embodiment of this application, the trajectory generation module 330 is specifically used to: update the vehicle's trajectory at the next moment in response to the driving trajectory of the lateral token and the driving trajectory of the longitudinal token.

[0087] The driving trajectory of the vehicle at the next moment is determined by the driving trajectories of the lateral token and the longitudinal token, respectively.

[0088] In one embodiment of this application, the horizontal token prediction module 310 is further configured to:

[0089] The curvature and speed of the vehicle at the current moment are used as inputs to the lateral planning problem of the vehicle. The rate of change of curvature of the predicted path, the length of the path, and whether the vehicle is reversing are used as discrete prediction variables. The prediction results of the lateral token are obtained by tokenizing the output.

[0090] In one embodiment of this application, the horizontal token prediction module 310 is further configured to:

[0091] Based on the prediction results of the lateral token and the calculation of the spiral line, the lateral token is converted into a path as the trajectory of the lateral token.

[0092] In one embodiment of this application, the vertical token prediction module 320 is further configured to:

[0093] The vehicle's current speed, acceleration, and the lateral token are used as inputs to the vehicle's longitudinal planning problem. The target speed, target acceleration, and time duration of the vehicle are predicted, and the result is tokenized to obtain the prediction result of the longitudinal token.

[0094] In one embodiment of this application, the vertical token prediction module 320 is further configured to:

[0095] Based on the prediction results of the vertical token and the Frenet coordinate system, the vertical token is converted into a path as the trajectory of the vertical token.

[0096] In one embodiment of this application, it further includes: an association module, used for

[0097] Establish the correlation between the horizontal token and the vertical token, such that the labeling result of the horizontal token at each time step affects the prediction result of the vertical token; and / or the labeling result of the vertical token at each time step inversely affects the prediction result of the horizontal token.

[0098] In one embodiment of this application, the trajectory generation module 330 is further configured to:

[0099] The vehicle's curvature and / or speed at the next moment are updated based on the driving trajectory of the longitudinal token.

[0100] The lateral token's trajectory updates the vehicle's speed and / or acceleration at the next moment.

[0101] It is understood that the above-mentioned action tagging device can implement each step of the action tagging method provided in the foregoing embodiments. The relevant explanations of the action tagging method are applicable to the action tagging device and will not be repeated here.

[0102] Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Please refer to it. Figure 4 At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and memory. The memory may include main memory, such as high-speed random-access memory (RAM), or non-volatile memory, such as at least one disk drive. Of course, the electronic device may also include other hardware required for other business operations.

[0103] The processor, network interface, and memory can be interconnected via an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 4 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0104] Memory is used to store programs. Specifically, programs may include program code, which includes computer operation instructions. Memory may include main memory and non-volatile memory, and provides instructions and data to the processor.

[0105] The processor reads the corresponding computer program from non-volatile memory into main memory and then executes it, forming an action tokenization device at the logical level. The processor executes the program stored in memory and specifically performs the following operations:

[0106] Predict the lateral token based on the vehicle's curvature and speed at the current moment;

[0107] Based on the lateral token, the vehicle's current speed and acceleration, predict the longitudinal token;

[0108] In response to the driving trajectory of the lateral token and the driving trajectory of the longitudinal token, the vehicle's driving trajectory at the next moment is updated.

[0109] The above is as stated in this application. Figure 2The method for executing the action tagging device disclosed in the illustrated embodiments can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly manifested as execution by a hardware decoding processor, or execution by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0110] The electronic device can also perform Figure 2 A method for executing an action tagging device, and implementation of the action tagging device in... Figure 2 The functions of the embodiments shown are not described in detail here.

[0111] This application also proposes a computer-readable storage medium that stores one or more programs, the programs including instructions that, when executed by an electronic device including multiple applications, enable the electronic device to perform... Figure 2 The method executed by the action tagging device in the illustrated embodiment is specifically used to perform:

[0112] Predict the lateral token based on the vehicle's curvature and speed at the current moment;

[0113] Based on the lateral token, the vehicle's current speed and acceleration, predict the longitudinal token;

[0114] In response to the driving trajectory of the lateral token and the driving trajectory of the longitudinal token, the vehicle's driving trajectory at the next moment is updated.

[0115] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0116] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0117] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0118] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0119] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0120] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0121] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0122] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0123] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0124] The above description is merely an embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of this application should be included within the scope of the claims of this application.

Claims

1. An action tokenization method, wherein, The method includes: Predict the lateral token based on the vehicle's curvature and speed at the current moment; Based on the lateral token, the vehicle's current speed and acceleration, predict the longitudinal token; In response to the driving trajectory of the lateral token and the driving trajectory of the longitudinal token, the vehicle's driving trajectory at the next moment is updated.

2. The method as described in claim 1, wherein, The prediction of the lateral token based on the vehicle's curvature and speed at the current moment includes: The curvature and speed of the vehicle at the current moment are used as inputs to the lateral planning problem of the vehicle. The rate of change of curvature of the predicted path, the length of the path, and whether the vehicle is reversing are used as discrete prediction variables. The prediction results of the lateral token are obtained by tokenizing the output.

3. The method as described in claim 2, wherein, The method of predicting the lateral token based on the vehicle's curvature and speed at the current moment also includes: Based on the prediction results of the lateral token and the spiral formula, the lateral token is converted into a path as the trajectory of the lateral token.

4. The method of claim 1, wherein, The step of predicting the longitudinal token based on the lateral token, the vehicle's current speed, and acceleration includes: The vehicle's current speed, acceleration, and the lateral token are used as inputs to the vehicle's longitudinal planning problem. The target speed, target acceleration, and time duration of the vehicle are predicted, and the result is tokenized to obtain the prediction result of the longitudinal token.

5. The method of claim 4, wherein, The step of predicting the longitudinal token based on the lateral token, the vehicle's current speed, and acceleration also includes: Based on the prediction result of the vertical token, the vertical token is converted into a path as the trajectory of the vertical token.

6. The method of claim 1, wherein, The method further includes: Establish the correlation between the horizontal token and the vertical token, such that the labeling result of the horizontal token at each time step affects the prediction result of the vertical token; and / or the labeling result of the vertical token at each time step inversely affects the prediction result of the horizontal token.

7. The method of claim 1, wherein, In response to the driving trajectory (path) of the lateral token and the driving trajectory (trajectory) of the longitudinal token, update the vehicle's driving trajectory at the next moment, including: The vehicle's path is updated based on the lateral token's travel trajectory; Update the vehicle's trajectory based on the sum and path of the vertical tokens.

8. An action marking device, wherein, The device includes: The lateral token prediction module is used to predict the lateral token based on the vehicle's curvature and speed at the current moment. The longitudinal token prediction module is used to predict the longitudinal token based on the lateral token, the vehicle's speed and acceleration at the current moment; The trajectory generation module is used to update the vehicle's trajectory at the next moment in response to the driving trajectory of the lateral token and the driving trajectory of the longitudinal token.

9. An electronic device, comprising: processor; as well as A memory configured to store computer-executable instructions, which, when executed, cause the processor to perform the method of any one of claims 1 to 7.

10. A computer-readable storage medium storing one or more programs, which, when executed by an electronic device including a plurality of applications, cause the electronic device to perform the method of any one of claims 1 to 7.

Images