Driving behavior semantic library construction method, scenario-based retrieval method and system for autonomous driving

Abstract

The application belongs to the technical field of automatic driving, and particularly relates to a driving behavior semantic library construction method, a scenario-based retrieval method and a system for automatic driving. The driving behavior semantic library constructed by taking the codebook of a trajectory segmenter as a driving behavior semantic dictionary fully taps the potential value of the codebook and solves the problem of insufficient semantic utilization in the prior art. By preprocessing, cutting and converting massive historical driving data into compact Token sequences, storage overhead is greatly reduced, while key behavior semantics are retained, and efficient utilization of historical data is realized. The semantic library based on multi-level indexing supports fast scenario-based retrieval, so that the system can retrieve the processing strategy of a similar historical scenario in real time when facing rare corner cases, and provide reliable online decision assistance for an end-to-end model. The retrieval result can be directly used to automatically generate diversified simulation test cases, changing the situation of inefficient and limited coverage of traditional manual design.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, specifically to a method for constructing a semantic database of driving behavior for autonomous driving, a scenario-based retrieval method, and a system. Background Technology

[0002] Autonomous driving technology, as an important application area of ​​artificial intelligence, has made significant progress in recent years. It has evolved from early rule-based modular architectures to the current end-to-end deep learning architectures. End-to-end models directly map sensor inputs to vehicle control outputs, greatly simplifying the system architecture and improving overall optimization efficiency.

[0003] However, end-to-end models face numerous challenges in practical deployment, one of which is how to effectively utilize massive amounts of historical driving data and how to handle rare corner case scenarios in the training data. Traditional data storage and retrieval methods are insufficient to meet the real-time and accuracy requirements of autonomous driving systems.

[0004] Action Tokenizer is a key component in end-to-end autonomous driving models. Its core function is to discretize continuous trajectory data into a finite set of tokens. This technology is typically implemented based on Vector Quantization (VQ) or Variational Autoencoder (VAE) architectures.

[0005] During training, the Action Tokenizer learns a codebook containing several discrete vectors (called codewords). Each codeword represents a typical driving behavior pattern, such as "driving at a constant speed," "decelerating and following," or "changing lanes and overtaking." By encoding continuous trajectories into a sequence of tokens, efficient representation and compression of driving behavior can be achieved.

[0006] Although Action Tokenizer technology has been successfully applied in end-to-end autonomous driving models, existing technical solutions have the following main problems:

[0007] (1) Codebook semantics are not fully utilized: Existing technologies mainly use codebooks as intermediate representations within the model, failing to fully explore their potential value as a semantic dictionary of driving behavior.

[0008] (2) Low efficiency of historical data utilization: The massive amount of historical driving data is stored in the form of raw sensor data, which occupies a lot of storage space and is difficult to retrieve and utilize efficiently.

[0009] (3) Insufficient Corner Case processing capability: For rare scenarios, end-to-end models often lack sufficient training data to support them, resulting in reduced decision reliability, while existing technologies lack effective historical similar scenario retrieval mechanisms.

[0010] (4) Difficulty in generating simulation test cases: Offline simulation testing requires a large number of diverse test scenarios. Traditional methods rely on manual design, which is inefficient and has limited coverage. Summary of the Invention

[0011] In view of this, the purpose of this application is to provide a method for constructing a semantic database of driving behavior for autonomous driving, a scenario-based retrieval method and system, in order to solve the problems in the background art.

[0012] To achieve the above objectives, this application adopts the following technical solution:

[0013] The method for constructing a driving behavior semantic library for autonomous driving in this application includes the following steps:

[0014] The vehicle's driving data samples at multiple historical time points are acquired. The driving data samples include trajectory samples, perception data samples, and map data samples. The trajectory samples include the position, speed, acceleration, and heading angle of multiple trajectory points.

[0015] Driving data samples from multiple historical time points are preprocessed to obtain preprocessed driving data from multiple historical time points. The preprocessing includes timestamp alignment, outlier removal, coordinate system unification, and normalization.

[0016] Preprocessed trajectory data is extracted from the preprocessed driving data, and the preprocessed trajectory data from multiple historical time points is segmented to obtain multiple trajectory segment samples.

[0017] The trajectory segment samples are converted into token sequences based on a pre-built trajectory segmenter;

[0018] The context features of the Token sequence are extracted from the preprocessed driving data, and a structured Token sequence is constructed based on the Token sequence and the context features of the Token sequence.

[0019] The structured token sequence is semantically annotated to obtain a structured token sequence with semantic tags; and an index structure of the structured token sequence with semantic tags is constructed to obtain a semantic library, wherein the index structure includes an inverted index, a vector index, and a classification index.

[0020] In one embodiment of this application, preprocessed trajectory data from multiple historical time points are segmented to obtain multiple trajectory segment samples, including:

[0021] Preprocessed trajectory data from multiple historical time points are segmented using a pre-built sliding window to obtain multiple trajectory segment samples, which include historical trajectories and future trajectories of a target duration for the historical trajectories.

[0022] In one embodiment of this application, the trajectory segmenter includes an encoder for mapping segments to latent space vector sequences, a codebook for converting latent space vector sequences into token sequences, and a decoder for mapping the token sequences to trajectory decoding segments. The method for constructing the trajectory segmenter includes:

[0023] Obtain driving trajectory segment samples;

[0024] The trajectory segment sample is input into the encoder to obtain the predicted token sequence output by the codebook and the trajectory decoded segment output by the decoder;

[0025] The loss is calculated based on a pre-constructed loss function, the driving trajectory segment samples, the trajectory decoding segments, and the predicted token sequence, wherein the mathematical expression of the loss function is:

[0026]

[0027] In the formula, Indicates loss, This represents a sample of a driving trajectory segment. This represents a trajectory decoding segment. This represents the sequence of latent space vectors output by the encoder. This indicates that the gradient operation is stopped. For the codeword sequence in the codebook, The index is the token sequence. For loss balancing parameters;

[0028] The trajectory segmenter is trained based on the loss.

[0029] In one embodiment of this application, training the trajectory segmenter based on the loss includes:

[0030] In the first stage, the codebook is frozen, and the encoder and decoder are trained based on the loss.

[0031] At the end of the first stage, the second stage begins and the codebook is decoded. The encoder, the codebook, and the decoder are trained based on the loss until training is complete.

[0032] In one embodiment of this application, extracting the contextual features of the Token sequence from the preprocessed driving data includes:

[0033] The contextual features of the token sequence are obtained by extracting perception data, high-precision map data, and vehicle status data that are time-aligned with the token sequence from the preprocessed driving data.

[0034] In one embodiment of this application, semantic annotation is performed on the structured token sequence to obtain a structured token sequence with semantic tags, including:

[0035] The structured token sequences are clustered to obtain multiple clusters; and coarse classification labels are assigned to the structured token sequences within each cluster.

[0036] Assign fine-classification labels to the structured token sequences within each cluster, and perform semantic annotation based on the coarse-classification labels and fine-classification labels of each structured token sequence to obtain structured token sequences with semantic labels.

[0037] This application also provides a system for building a driving behavior semantic library for autonomous driving, including:

[0038] The first acquisition module is used to acquire driving data samples of the vehicle at multiple historical time points. The driving data samples include trajectory samples, perception data samples and map data samples. The trajectory samples include the position, speed, acceleration and heading angle of multiple trajectory points.

[0039] The preprocessing module is used to preprocess driving data samples from multiple historical time points to obtain preprocessed driving data from multiple historical time points. The preprocessing includes timestamp alignment, outlier removal, coordinate system unification, and normalization.

[0040] The segmentation module is used to extract preprocessed trajectory data from the preprocessed driving data and segment the preprocessed trajectory data at multiple historical time points to obtain multiple trajectory segment samples.

[0041] The first word segmentation module is used to convert the trajectory fragment samples into a token sequence based on a pre-built trajectory word segmenter;

[0042] The structured processing module is used to extract the context features of the Token sequence from the preprocessed driving data, and construct a structured Token sequence based on the Token sequence and the context features of the Token sequence.

[0043] The semantic library construction module is used to perform semantic annotation on the structured token sequence to obtain a structured token sequence with semantic tags; and to construct an index structure for the structured token sequence with semantic tags to obtain a semantic library, wherein the index structure includes an inverted index, a vector index, and a classification index.

[0044] This application also provides a contextualized retrieval method for a driving behavior semantic database for autonomous driving, including:

[0045] The system acquires the historical trajectory segment of the current vehicle for the first target time period before the current time period, the predicted trajectory segment for the second target time period after the current time point, and the perception data and map data of the current vehicle. The predicted trajectory segment is generated by the end-to-end model of the current vehicle.

[0046] The historical trajectory fragment and the predicted trajectory fragment are concatenated to form the current scene trajectory fragment, and the current scene trajectory fragment is converted into the current token sequence based on the pre-built trajectory segmenter;

[0047] Extract current context features from the current scene trajectory segment, the perception data, and the map data;

[0048] Based on the current token sequence and the current context features, a weighted voting-based retrieval is performed in the semantic database to obtain the semantic label of the current token sequence;

[0049] Based on the current token sequence, the contextual features of the current token sequence, and the semantic tags of the current token sequence, a search is performed in the semantic database to obtain search results, wherein the search results are used for online decision support and simulation test case generation.

[0050] In one embodiment of this application, a search is performed in the semantic database based on the current token sequence, the contextual features of the current token sequence, and the semantic tags of the current token sequence to obtain search results, including:

[0051] Based on the inverted index in the semantic library, a first candidate set that locally matches the current token sequence is retrieved; based on the vector index in the semantic library, a second candidate set that matches the contextual features of the current token sequence is retrieved; and based on the classification index in the semantic library, a third candidate set that matches the semantic tags of the current token sequence is retrieved.

[0052] Merge the first candidate set, the second candidate set, and the third candidate set to obtain the candidate set;

[0053] Calculate the similarity between multiple structured token sequences in the candidate set and the current token sequence, wherein the similarity includes token sequence similarity, context feature similarity, and semantic label similarity;

[0054] The K most similar structured token sequences are output as search results.

[0055] This application also provides a contextualized retrieval system for a semantic database of driving behavior for autonomous driving, including:

[0056] The second acquisition module is used to acquire the historical trajectory segment of the current vehicle for the first target time before the current time period, the predicted trajectory segment for the second target time after the current time point, and to acquire the perception data and map data of the current vehicle. The predicted trajectory segment is generated by the end-to-end model of the current vehicle.

[0057] The second word segmentation module is used to concatenate the historical trajectory fragment and the predicted trajectory fragment into the current scene trajectory fragment, and convert the current scene trajectory fragment into the current token sequence based on the pre-built trajectory word segmenter;

[0058] The feature extraction module is used to extract current context features from the current scene trajectory segment, the perception data, and the map data;

[0059] The preliminary retrieval module is used to perform a weighted voting-based retrieval in the semantic database based on the current token sequence and the current context features to obtain the semantic tags of the current token sequence;

[0060] The final retrieval module is used to perform a retrieval in the semantic database based on the current token sequence, the context features of the current token sequence, and the semantic tags of the current token sequence, and obtain retrieval results, wherein the retrieval results are used for online decision support and simulation test case generation.

[0061] The beneficial effects of this application are as follows: The driving behavior semantic library construction method, scenario-based retrieval method, and system for autonomous driving constructed in this application utilize the codebook of the trajectory segmenter as a driving behavior semantic dictionary, fully exploring the potential value of the codebook and solving the problem of insufficient semantic utilization in existing technologies. By preprocessing, segmenting, and converting massive historical driving data into compact structured token sequences, storage overhead is significantly reduced while retaining key behavioral semantics, achieving efficient utilization of historical data. The semantic library based on multi-level indexes (inverted index, vector, classification) supports rapid scenario-based retrieval, enabling the system to retrieve processing strategies for similar historical scenarios in real time when facing rare corner cases, providing reliable online decision support for end-to-end models and significantly improving decision reliability in rare scenarios. In addition, the retrieval results can be directly used to automatically generate diverse simulation test cases, changing the situation of inefficient and limited coverage of traditional manual design, and greatly improving testing efficiency and scenario coverage. This application has significant improvements in semantic representation, data compression, real-time retrieval, decision support, and simulation generation. Attached Figure Description

[0062] The present application will be further described below with reference to the accompanying drawings and embodiments:

[0063] Figure 1 This is a flowchart illustrating a method for constructing a driving behavior semantic library for autonomous driving, as shown in one embodiment of this application;

[0064] Figure 2 This is a flowchart illustrating a scenario-based retrieval method for a driving behavior semantic database for autonomous driving, as shown in one embodiment of this application.

[0065] Figure 3 This is a structural diagram of a driving behavior semantic library construction system for autonomous driving, as shown in one embodiment of this application;

[0066] Figure 4 This is a structural diagram of a contextualized retrieval system for a semantic database of driving behavior for autonomous driving, as shown in one embodiment of this application. Detailed Implementation

[0067] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that, unless otherwise specified, the following embodiments and features in the embodiments can be combined with each other.

[0068] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this application. Therefore, the drawings only show the layers related to this application and are not drawn according to the actual number, shape and size of the layers in the actual implementation. In the actual implementation, the form, number and proportion of each layer can be arbitrarily changed, and the layer layout may also be more complex.

[0069] Numerous details are explored in the following description to provide a more thorough explanation of embodiments of this application; however, it will be apparent to those skilled in the art that embodiments of this application may be practiced without these specific details.

[0070] This application utilizes the codebook of the Action Tokenizer in the end-to-end autonomous driving model as a semantic dictionary for driving behavior, constructs a massive semantic library for driving behavior, and implements a fast retrieval and decision-making assistance technology for corner cases based on this library.

[0071] This application aims to achieve the following technical objectives:

[0072] (1) Constructing a driving behavior semantic library: Using the codebook of the trained trajectory segmenter as a driving behavior semantic dictionary, the historical driving data is automatically segmented and encoded into a compact token sequence to construct a structured driving behavior semantic library.

[0073] (2) Achieve efficient scene retrieval: When a vehicle encounters a Corner Case, it can quickly retrieve the processing strategies corresponding to similar token sequences in history, providing auxiliary support for online decision-making.

[0074] (3) Support for simulation test case generation: Based on the retrieved similar scenario token sequences, offline simulation test cases are automatically generated to improve test coverage and efficiency.

[0075] (4) Improve the efficiency of massive data utilization: By using tokenization compression, the storage requirements of historical data are greatly reduced, while retaining key behavioral semantic information.

[0076] Figure 1 This is a flowchart illustrating a method for constructing a driving behavior semantic library for autonomous driving, as shown in one embodiment of this application. Figure 1 As shown, the method for constructing a driving behavior semantic library for autonomous driving in this application mainly includes the following steps:

[0077] S110, acquire vehicle driving data samples at multiple historical time points, wherein the driving data samples include trajectory samples, perception data samples and map data samples, and the trajectory samples include the position, speed, acceleration and heading angle of multiple trajectory points;

[0078] The trajectory samples are described below:

[0079] Specifically, each trajectory sample corresponds to a historical time point (sampling time), and records the following 5 dimensions of state variables:

[0080] (1) Position: The horizontal coordinate x and vertical coordinate y of the vehicle in a global coordinate system (such as UTM) or a local relative coordinate system, with an accuracy of 0.1 meters.

[0081] (2) Speed: Instantaneous vehicle speed v, in m / s, provided by wheel speed gauge or inertial navigation system.

[0082] (3) Acceleration: The longitudinal acceleration of the vehicle, a, in m / s², can be obtained directly through velocity differential or IMU.

[0083] (4) Heading angle: The angle between the vehicle's heading and the tangent of the due north direction or the center line of the lane. The unit is radians or degrees.

[0084] The sensory data samples are described below:

[0085] Each perception data sample corresponds to a structured understanding of the vehicle's surrounding environment at a specific historical point in time, obtained through existing perception models, and mainly includes:

[0086] Table 1. Examples of Perceptual Data Sample Content

[0087]

[0088] The map data sample is introduced as follows:

[0089] Each map data sample records high-precision map information near the vehicle's location, typically aligned with trajectory samples based on spatiotemporal correlation, as shown in the table below:

[0090] Table 2. Example of map data sample content

[0091]

[0092] S120, preprocess driving data samples from multiple historical time points to obtain preprocessed driving data from multiple historical time points, wherein the preprocessing includes timestamp alignment, outlier removal, coordinate system unification, and normalization.

[0093] Specifically, the preprocessing process includes:

[0094] (1) Timestamp Alignment: Due to the different original sampling frequencies of trajectory samples (from GNSS / IMU, typically 100Hz), perception data samples (from camera / LiDAR, typically 10-30Hz), and map data samples (query matching results), they need to be unified to the same time base. Specifically, this includes:

[0095] (1-1) Using GPS time or vehicle system clock as a unified time reference, select a fixed target sampling period. (For example =0.1 seconds, or 10 Hz).

[0096] (1-2) For each target time point The position, velocity, acceleration, and heading angle at that moment can be obtained from the original trajectory data through linear interpolation.

[0097] (1-3) For sensing data (such as obstacle position and speed), if there is a sensor sampling result at the target time point, it is used directly; otherwise, nearest neighbor or linear interpolation (for continuously changing quantities) is used to fill the data.

[0098] (1-4) For map data, since its static attributes do not change continuously over time, the map tiles that match the vehicle location at the target time point can be directly used without interpolation.

[0099] (1-5) After alignment, each historical time point Each has complete trajectory points, perception data, and map data, forming a sample with a uniform sampling rate.

[0100] (2) Outlier removal

[0101] Remove abnormal data points caused by sensor noise, communication packet loss, or physical reasons. Specifically, this includes:

[0102] (2-1) Trajectory anomaly removal:

[0103] Calculate the displacement at two adjacent time points ,like (in The maximum physical speed of the vehicle. If the tolerance margin is exceeded, the position jump is determined to be abnormal, and the trajectory point is removed.

[0104] Calculate acceleration ,like ( If the maximum acceleration is reached, then that point is eliminated.

[0105] Using the Hampel filter: Take a sliding window (window length 5-11 points) for the velocity sequence and calculate the median within the window. and median of absolute deviation If the speed at a certain point If it is an anomaly, it is marked as an anomaly and replaced with the median.

[0106] (2-2) Sensory anomaly removal:

[0107] If an obstacle's speed exceeds the physical limit (e.g., 200 km / h) or it exhibits negative mass, overlapping bounding boxes, etc., the obstacle information is discarded directly.

[0108] If the number of obstacles exceeds the normal range at the same time (e.g., more than 100 on city roads), it may be a false detection by the sensor. The entire frame of perception data will be marked as unreliable and removed.

[0109] (2-3) Map data verification:

[0110] Check if the map data matches the vehicle's location (e.g., the distance from the lane centerline is >5 meters). If they do not match, query again or discard the sample.

[0111] For single-point anomalies, if they can be repaired (e.g., speed anomaly but position is normal), interpolation or filtering is used for repair; if there are continuous large segments of anomalies, the data segment is discarded directly.

[0112] (3) Coordinate System 1:

[0113] Transforming coordinate data from different sensors and maps to the same spatial reference frame eliminates differences in absolute coordinate systems, including:

[0114] First, all data is unified to a global coordinate system (such as UTM, Universal Transverse Mercator projection). The positions in the trajectory samples may originally be WGS84 latitude and longitude, which are converted to UTM metric coordinates through projection.

[0115] The location of obstacles in the perception data is usually given as relative coordinates with the vehicle as the origin. It needs to be converted to UTM coordinates in combination with the vehicle's global pose (position + heading).

[0116] The map data itself is stored in the UTM coordinate system and can be used directly.

[0117] After achieving global unification, to eliminate scale differences caused by different road sections and driving directions, the system is then transformed into a local Flyner coordinate system: with the center of the rear axle of the vehicle at the current moment as the origin, the direction of the vehicle's front as the positive longitudinal axis, and the rightward direction as the positive transverse axis. All trajectory points and obstacle positions are represented in this local coordinate system.

[0118] When performing coordinate transformation, it is important to maintain time consistency: the vehicle's pose should be the interpolation result of the current sampling point.

[0119] (4) Normalization process:

[0120] Scaling continuous numerical features to bring them into a similar dimensional range facilitates subsequent distance calculations and model training. This includes:

[0121] Z-score normalization (applicable to position, velocity, acceleration, etc.):

[0122] For each feature (e.g., velocity v), calculate its mean across all historical samples. and standard deviation Then convert:

[0123]

[0124] These are the features after standardization.

[0125] For location coordinates, x and y can be standardized separately, or normalized to the road width in a local coordinate system (e.g., divided by the lane width).

[0126] Extreme value normalization (applicable to angle and percentage features):

[0127] For heading angle , can be converted and The two components are then standardized separately. The probability of the perceived presence of obstacles is normalized to [0,1] using Min-Max.

[0128] Category feature encoding: Category variables such as obstacle type, lane line type, and traffic sign type are encoded using one-hot encoding or embedding vectors (offline pre-training) and are not directly normalized as continuous numerical values.

[0129] Important notes on time series normalization: To avoid future information leakage, the normalization parameters (mean and standard deviation) are only obtained from the training set or historical data and are fixed and saved during preprocessing, and then applied to all samples.

[0130] S130, extract preprocessed trajectory data from the preprocessed driving data, and segment the preprocessed trajectory data at multiple historical time points to obtain multiple trajectory segment samples;

[0131] In this application, a sliding window mechanism is used to segment continuous driving data into trajectory segments of fixed length. The window length is determined based on the codebook's representation capability, typically taking 2-5 seconds of historical trajectory and 3-8 seconds of future trajectory.

[0132] S140, the trajectory segment sample is converted into a token sequence based on a pre-built trajectory segmenter;

[0133] This application inputs each trajectory segment into the encoder of the trajectory segmenter, obtains the latent space representation, and then searches for the nearest codeword through the codebook to obtain the corresponding Token ID sequence.

[0134] In one embodiment of this application, the trajectory segmenter includes an encoder for mapping segments to latent space vector sequences, a codebook for converting latent space vector sequences into token sequences, and a decoder for mapping the token sequences to trajectory decoding segments. The method for constructing the trajectory segmenter includes:

[0135] (1) Obtain driving trajectory segment samples;

[0136] The trajectory segmenter needs to learn to discretize continuous driving behavior into a finite number of typical patterns (code words). Therefore, the training data must cover diverse driving scenarios and contain complete dynamic processes. Each trajectory segment sample consists of continuous trajectory points within a fixed duration (e.g., 2 seconds of historical data + 5 seconds of future data). Each point contains motion states such as position, velocity, acceleration, and heading angle.

[0137] (2) Input the trajectory segment sample into the encoder to obtain the predicted token sequence output by the codebook and the trajectory decoding segment output by the decoder;

[0138] Specifically, the three sub-modules work together as follows:

[0139] Encoder: Employs multi-layer Transformer or convolutional networks to process the input trajectory segments. Mapped to a sequence of vectors in the latent space The encoder's role is to extract high-order features of the trajectory and compress them into a low-dimensional manifold.

[0140] Quantization module: For each latent vector In the codebook The most similar codeword is found through nearest neighbor search:

[0141]

[0142] in This is the **Token** for that time step. The tokens from all time steps form the predicted token sequence. The quantization process forces continuous and diverse latent vectors to be mapped onto a finite number of discrete prototypes, compelling the model to learn typical behavioral patterns.

[0143] Decoder: Receives the quantized vector sequence (That is, the codeword corresponding to the token), and reconstructs the trajectory segment with the same dimension as the input through another set of neural networks (such as Transformer or MLP). The goal of the decoder is to make the reconstructed trajectory as close as possible to the original input, thereby ensuring that the token sequence retains the core information of the original trajectory.

[0144] (3) Calculate the loss based on the pre-constructed loss function, the driving trajectory segment samples, the trajectory decoding segments, and the predicted token sequence, wherein the mathematical expression of the loss function is:

[0145]

[0146] In the formula, Indicates loss, This represents a sample of a driving trajectory segment. This represents a trajectory decoding segment. This represents the sequence of latent space vectors output by the encoder. This indicates that the gradient operation is stopped. For the codeword sequence in the codebook, The index is the token sequence. For loss balancing parameters;

[0147] The reconstruction loss is defined as the loss that forces the decoder's output trajectory to be geometrically and kinematically consistent with the input trajectory. By minimizing the mean squared error, the token sequence (i.e., the indices of the selected codewords) must contain enough information to recover the original trajectory. This is fundamental to ensuring that the discrete representation does not lose key behavioral features.

[0148] This term represents the codebook loss, which reduces the codeword vectors in the codebook. The latent vector output to the encoder Close to. Due to Indicates to The gradient is stopped; this loss only updates the codebook parameters and does not affect the encoder. By continuously adjusting the codewords, each codeword corresponding to a token can represent the latent space center of its region, thus forming a semantically clear dictionary of atomic behaviors.

[0149] This represents the commitment loss, which imposes a constraint on the encoder, causing its output latent vector to... Don't stray too far from your chosen code. This indicates that gradients are stopped for codewords, therefore this loss only updates the encoder. Parameters (Typically set to 0.25) This balances the encoder's degrees of freedom and quantization error. Without this, the encoder might arbitrarily map latent vectors to regions far from the codeword, resulting in severe information loss after quantization.

[0150] (4) The trajectory segmenter is trained based on the loss.

[0151] Specifically, in the first stage, the codebook is frozen, and the encoder and the decoder are trained based on the loss;

[0152] At the end of the first stage, the second stage begins and the codebook is decoded. The encoder, the codebook, and the decoder are trained based on the loss until training is complete.

[0153] In the first stage, at the start of training, the codebook is randomly initialized and completely mismatched with the true latent space distribution. Immediately allowing the codebook to learn from the encoder output could cause the codewords to get stuck in local optima, or even result in a large number of "dead codewords" (codewords that were never selected). Without codebook intervention, the encoder and decoder learn how to compress trajectories into the latent space and recover them through a pure reconstruction task (similar to a regular autoencoder). At this point, the latent vectors... It can move freely, unconstrained by quantization, thus quickly forming an effective continuous representation of the input trajectory. This provides a good initial latent space for subsequent quantization learning, preventing the codebook from degrading in chaotic early updates.

[0154] In the second stage, after the first stage ends (usually marked by the reconstruction error decreasing to a stable state or reaching a preset number of rounds), the codebook is set to a trainable state. At this point, all three modules (encoder, codebook, and decoder) participate in optimization simultaneously. After sufficient joint training, the three losses reach a balance: small reconstruction error (good trajectory fidelity), small quantization error (high precision of discrete representation), and high codebook utilization (no dead codewords). The resulting trajectory segmenter compresses the trajectory information into a discrete token sequence while simultaneously restoring the original trajectory with high quality.

[0155] S150, extract the context features of the Token sequence from the preprocessed driving data, and construct a structured Token sequence based on the Token sequence and the context features of the Token sequence;

[0156] Extract contextual information corresponding to the trajectory segments, including: environmental perception data (obstacle location, speed, type), high-precision map information (lane lines, traffic signs, intersection structure), and vehicle status (speed, acceleration, heading angle). Specifically, this includes:

[0157] The contextual features of the token sequence are obtained by extracting perception data, high-precision map data, and vehicle status data that are time-aligned with the token sequence from the preprocessed driving data.

[0158] The token sequence only describes the trajectory of the vehicle itself (position, speed, heading, etc.), but the same behavior (such as "deceleration") has different meanings in different environments—there is a car ahead, there is a red light, or it is just a change in road conditions, the decision-making logic and safety risks are completely different.

[0159] Therefore, perception data (obstacle location / speed / type), high-precision map data (lane lines / intersections / signs), and vehicle state data (speed / acceleration / heading angle) that are strictly aligned with the token sequence in time are extracted from the preprocessed data as the "contextual features" of the token sequence.

[0160] Packing the token sequence together with contextual features creates a structured token sequence: the former provides behavioral semantics, and the latter provides environmental semantics; the two complement each other to describe a complete driving scenario.

[0161] S160, Semantically annotate the structured token sequence to obtain a structured token sequence with semantic tags; and construct an index structure for the structured token sequence with semantic tags to obtain a semantic library, wherein the index structure includes an inverted index, a vector index, and a classification index.

[0162] Specifically, based on the distribution characteristics and contextual information of the token sequence, semantic tags, such as "following a vehicle," "turning left at an intersection," and "emergency avoidance," are automatically or semi-automatically added to the token sequence. Taking semi-automatic annotation as an example, the annotation process of this application includes:

[0163] S161, Cluster the structured token sequences to obtain multiple clusters; and assign coarse classification labels to the structured token sequences within each cluster;

[0164] Specifically, a hierarchical clustering method based on shape-based dynamic time warping (DTW) distance is used to automatically form multiple clusters from a massive number of structured token sequences (each sequence already contains a TokenID sequence and aligned contextual features). The token sequences within each cluster are highly similar in terms of behavioral trajectory morphology. Then, each cluster is automatically assigned a coarse classification label (such as "following other vehicles", "left turn", "lane keeping").

[0165] S162, assign fine classification labels to the structured token sequences within each cluster, and perform semantic annotation based on the coarse and fine classification labels of each structured token sequence to obtain structured token sequences with semantic labels.

[0166] Within each coarse cluster, further secondary clustering or rule matching is performed using contextual features (perception, map, vehicle status) to subdivide the cluster into several subclusters. For example, the coarse cluster "following the car" can be subdivided into "close following," "safe following," and "gradually approaching following" based on features such as distance to the car in front, relative speed, and headway. Finally, a combined label is assigned to each structured token sequence: [coarse classification label] - [fine classification label] (e.g., "following the car - close following").

[0167] Finally, a multi-level index structure is established, including: an inverted index based on token sequences, a vector index based on contextual features, and a classification index based on semantic labels. Specifically, this includes:

[0168] (1) Inverted index

[0169] Each token sequence is divided into several short segments (N-grams), and each segment is recorded in which sequences it appears in, so that candidates containing the same subsequence can be quickly located during retrieval.

[0170] Construction method:

[0171] (1-1) For each token sequence, slide the segment with a fixed window length (e.g., 3) to obtain multiple N-gram terms.

[0172] (1-2) Create a hash table: the key is N-gram, and the value is a list of all sequence IDs containing the N-gram.

[0173] (1-3) Only the sequence ID is saved during storage, and the specific weight is not saved, for subsequent accurate calculation.

[0174] (2) Vector index

[0175] By mapping context feature vectors to a neighbor graph or partitioned region in a high-dimensional space, a fast approximate nearest neighbor search can be achieved.

[0176] Construction method:

[0177] (2-1) Extract the context features (such as obstacle distance, lane curvature, vehicle speed, etc.) of each structured token sequence to form a vector of fixed dimensions.

[0178] (2-2) Use HNSW (Hierarchical Navigable Small World Graph) or IVF (Inverted File) algorithms: Insert all vectors into the graph structure or cluster centers to establish a fast route from the query vector to the candidate vector.

[0179] (2-3) Store the index file and load it into memory for retrieval.

[0180] (3) Category Index

[0181] Using semantic tags as keys, it directly maps to all sequence IDs that have that tag, supporting fast filtering of hierarchical tags.

[0182] Construction method:

[0183] (3-1) Construct a label tree: The root node is "All", the first level is coarse category labels (such as "Left turn"), and the second level is fine category labels (such as "Left turn - Pedestrians present").

[0184] (3-2) For each sequence, add its ID to the list of nodes with corresponding semantic labels and all ancestor nodes.

[0185] (3-3) Storage method: hash table or bitmap, supporting fast set operations (union, intersection).

[0186] After completing the construction of the semantic database, searches can be performed based on it to facilitate online decision-making assistance and case generation for vehicles. Figure 2 This is a flowchart illustrating a scenario-based retrieval method for a driving behavior semantic database for autonomous driving, as shown in one embodiment of this application. Figure 2 As shown, the scenario-based retrieval method for driving behavior semantic database for autonomous driving in this application mainly includes the following steps:

[0187] S210, acquire the historical trajectory segment of the current vehicle for the first target time before the current time period, the predicted trajectory segment for the second target time after the current time point, and acquire the perception data and map data of the current vehicle, wherein the predicted trajectory segment is generated by the end-to-end model preset in the current vehicle;

[0188] The data structure and acquisition method of the trajectory segments, perception data, and map data are the same as those of the data samples introduced earlier, and will not be repeated here.

[0189] S220, the historical trajectory segment and the predicted trajectory segment are concatenated into the current scene trajectory segment, and the current scene trajectory segment is converted into the current token sequence based on the pre-built trajectory segmenter;

[0190] By using the trained trajectory segmenter, the trajectory fragments of the current scene are converted into the current token sequence, thus realizing the token serialization representation of the trajectory fragments.

[0191] S230, extract current context features from the current scene trajectory segment, the perception data, and the map data;

[0192] S240, based on the current token sequence and the current context features, a weighted voting-based retrieval is performed in the semantic database to obtain the semantic label of the current token sequence;

[0193] This application requires two searches: (1) performing an unlabeled preliminary search to determine the labels; and (2) a final search based on the token sequence, contextual semantic feature vector, and semantic labels.

[0194] In step S240, the system uses the token sequence (unlabeled) and contextual features of the current scene to calculate similarity in the semantic database and retrieve the Top-K most similar historical scenes. This initial retrieval does not rely on the semantic labels of the current scene, but is based solely on token sequence similarity and contextual feature similarity.

[0195] The system then retrieves K historical scenes, each with pre-annotated semantic tags (generated by the offline construction process). These K tags are statistically analyzed, and the tag with the highest frequency is selected as the candidate tag for the current scene. A weighted voting method is then used to obtain a comprehensive similarity score, and the tag with the highest comprehensive similarity score is selected as the semantic tag for the current scene.

[0196] S250, based on the current token sequence, the context features of the current token sequence, and the semantic tags of the current token sequence, a search is performed in the semantic database to obtain search results, wherein the search results are used for online decision support and simulation test case generation.

[0197] The final retrieval process is as follows:

[0198] S251, based on the inverted index in the semantic library, retrieve a first candidate set that locally matches the current token sequence; based on the vector index in the semantic library, retrieve a second candidate set that matches the contextual features of the current token sequence; and based on the classification index in the semantic library, retrieve a third candidate set that matches the semantic tags of the current token sequence.

[0199] This step utilizes three different indices to quickly retrieve candidate fragments from different dimensions, avoiding bias or omissions caused by a single index.

[0200] Inverted index recall (first candidate set) ):

[0201] The current token sequence Q is divided into fixed-length N-grams (e.g., length 3). Each N-gram is used as a query term to search for the historical fragment ID containing that term in the inverted index. All matching fragment IDs are merged and deduplicated to obtain... This candidate set covers segments that are similar to the current scene in local subsequences, and is particularly good at matching segments with the same behavioral patterns but slightly different overall lengths.

[0202] Vector Index Recall (Second Candidate Set) ):

[0203] Current context feature vector As a query, an approximate nearest neighbor search is performed in the vector index (such as HNSW) to retrieve the M nearest historical fragment IDs (e.g., M=500), forming... This candidate set covers historical fragments that are most similar to the current scenario in terms of environmental interaction, road structure, and vehicle status.

[0204] Category Index Recall (Third Candidate Set) ):

[0205] Based on the semantic tags of the current scenario (obtained from preliminary search voting), all historical fragment IDs with the same or similar tags (such as the same parent tag) are directly retrieved from the category index to obtain... This candidate set can quickly narrow the search scope to the same semantic category, avoiding invalid comparisons across categories.

[0206] S252, merge the first candidate set, the second candidate set, and the third candidate set to obtain a candidate set;

[0207] Perform a union operation on the three candidate sets to obtain the final candidate set. , ;

[0208] S253, calculate the similarity between the multiple structured token sequences in the candidate set and the current token sequence, wherein the similarity includes token sequence similarity, context feature similarity and semantic label similarity;

[0209] For candidate set Every historical fragment Calculate historical fragments Overall similarity with the current scene Overall similarity The mathematical expression is:

[0210]

[0211]

[0212]

[0213]

[0214] In the formula, The weighting coefficients for token sequence similarity. For token sequence similarity, These are the weighting coefficients for vector similarity. For vector similarity, The weighting coefficients for semantic label similarity. For semantic label similarity, The token sequence for the current scenario. For historical fragments Token sequence, Let be the length of the longest common subsequence of the two sequences. This is the context feature vector of the current token sequence. This is the context feature vector of the token sequence in the historical segment. For feature index, For the first The weights of each feature, The total dimension of the feature vector. The attenuation coefficient is... The semantic labels for the preceding token sequence. For the semantic tags of the token sequence of historical fragments, This represents the shortest distance between two semantic tags in the tree-like index structure of the semantic library.

[0215] S254 outputs the K most similar structured token sequences as the search results.

[0216] candidate set All fragments by Sort by highest to lowest, and take the top few. One (e.g.) =5 or The results are returned as the final search results. These results simultaneously meet the three conditions of behavioral similarity, environmental similarity, and semantic matching, and can be used for subsequent online decision support or offline simulation test case generation. Specific examples are as follows:

[0217] Online decision support methods

[0218] Based on the retrieved similar historical scenarios, auxiliary support is provided for current decision-making:

[0219] (1) Strategy recommendation: Extract successful processing strategies (such as trajectory, speed curve, control command) from historical scenarios as reference strategies for the current scenario.

[0220] (2) Risk assessment: Analyze the risk distribution in similar situations in historical scenarios to provide data support for risk assessment in the current scenario.

[0221] (3) Confidence calibration: When the confidence of the end-to-end model output is low, the model output is calibrated using the processing results of similar historical scenarios.

[0222] (4) Decision fusion: The historical strategies are fused with the end-to-end model output, and the fusion weights are dynamically adjusted based on the scene similarity and model confidence.

[0223] Offline simulation test case generation method

[0224] Automatically generate simulation test cases based on search results:

[0225] (1) Scene reconstruction: Using the token sequence and context features, the complete simulation scene is reconstructed through the decoder of the trajectory segmenter, including dynamic obstacle trajectories and static environment layout.

[0226] (2) Parameter perturbation: Based on the reconstructed scene, perturb the key parameters (such as obstacle speed and initial position) to generate variant test cases.

[0227] (3) Boundary exploration: For Corner Case scenarios, the boundary conditions of the scenario are explored through systematic parameter scanning to generate extreme test cases.

[0228] (4) Test case evaluation: Evaluate the effectiveness of the generated test cases and select test cases with high testing value to add to the test library.

[0229] The following is a specific implementation example of this application, including:

[0230] Implementation Environment

[0231] The implementation environment of this invention comprises two parts: a vehicle-mounted computing unit and a cloud server. The vehicle-mounted computing unit is deployed on an L4-level autonomous unmanned logistics vehicle and is responsible for real-time scene encoding and local retrieval; the cloud server is responsible for large-scale storage of the semantic database, index maintenance, and batch data processing.

[0232] Codebook Design

[0233] In this embodiment, the codebook design of the trajectory word segmenter is as follows:

[0234] (1) Codebook size: 8192 codewords, which can cover most common driving behavior patterns;

[0235] (2) Codeword dimension: 256-dimensional vector, which controls computational complexity while ensuring representational ability;

[0236] (3) Time granularity: Each token represents driving behavior within a 100ms time window;

[0237] (4) Spatial granularity: The trajectory coordinates adopt the vehicle local coordinate system with an accuracy of 0.1 meters.

[0238] semantic library size

[0239] In this embodiment, the scale of the semantic library construction is as follows:

[0240] (1) Data source: Actual operating data of 100 unmanned logistics vehicles over 6 months;

[0241] (2) Raw data volume: Approximately 500TB of raw sensor data;

[0242] (3) Data volume after tokenization: approximately 500GB of token sequences and context features, with a compression ratio of approximately 1000:1;

[0243] (4) Scene coverage: multiple scene types such as urban roads, highways, and park roads.

[0244] Search performance

[0245] In this embodiment, the performance metrics for scene retrieval are as follows:

[0246] (1) Search latency: The average local search latency on the vehicle is less than 50ms, which meets the real-time requirements;

[0247] (2) Search accuracy: The semantic relevance of the Top-5 search results exceeds 85%;

[0248] (3) Recall rate: For the labeled Corner Case scenario, the recall rate exceeds 90%.

[0249] The implementation process is as follows:

[0250] Trajectory segmenter training

[0251] (1) Dataset preparation: Collect 1 million high-quality driving trajectory segments, each trajectory containing 2 seconds of historical trajectory and 5 seconds of future trajectory;

[0252] (2) Model architecture: The improved VQ-VAE architecture is adopted, with a 6-layer Transformer encoder and an 8-layer Transformer decoder;

[0253] (3) Training strategy: Two-stage training is adopted. The first stage trains the encoder and decoder, and the second stage uses exponential moving average (EMA) to update the codebook.

[0254] (4) Evaluation indicators: reconstruction error (MSE) less than 0.05, codebook utilization rate exceeding 95%.

[0255] Semantic library construction process

[0256] (1) Data extraction: Extract historical driving data in batches from the data lake and organize it according to time windows;

[0257] (2) Parallel processing: Data is processed in parallel using the Spark distributed computing framework and encoded as a token sequence;

[0258] (3) Index building: Use Elasticsearch to build an inverted index and use FAISS to build a vector index;

[0259] (4) Quality verification: The encoding results are sampled and verified to ensure the accuracy and integrity of the token sequence.

[0260] Online search process

[0261] (1) Real-time encoding: The vehicle-side system performs token encoding on the current scene at a frequency of 10Hz;

[0262] (2) Local retrieval: Prioritize querying the semantic library subset cached locally on the vehicle to quickly obtain candidate results;

[0263] (3) Cloud query: For scenarios where no match is found locally, a search request is sent to the cloud;

[0264] (4) Results fusion: Combine local and cloud search results and return them in order of similarity.

[0265] The features of this application are as follows:

[0266] (1) Semantic representation: For the first time, the codebook of Action Tokenizer was used as a semantic dictionary of driving behavior, realizing the semantic representation of driving behavior;

[0267] (2) High-efficiency compression: Data compression is achieved through tokenization, with a compression ratio of over 1000:1;

[0268] (3) Multi-level retrieval: Supports multi-level retrieval strategies such as token sequences, context features, and semantic tags;

[0269] (4) Vehicle-to-cloud collaboration: Adopting a collaborative architecture that combines local vehicle retrieval with large-scale cloud retrieval;

[0270] (5) Closed-loop optimization: Supports closed-loop optimization for online decision support and offline simulation testing.

[0271] The beneficial effects of this application include:

[0272] (1) Improve Corner Case processing capability: By searching for similar historical scenarios, reliable decision-making references are provided for rare scenarios, significantly improving the system's ability to process Corner Cases;

[0273] (2) Reduce storage costs: Tokenization compression significantly reduces the need for historical data storage, and is expected to save more than 90% of storage costs;

[0274] (3) Improve simulation testing efficiency: Automatically generate diverse test cases, improving test case generation efficiency by more than 10 times;

[0275] (4) Enhanced system interpretability: The token sequence provides a semantic explanation of driving behavior, which enhances the interpretability of the end-to-end model;

[0276] (5) Accelerate model iteration: Based on the search results, target difficult scenarios to accelerate the iterative optimization of the end-to-end model.

[0277] like Figure 3 As shown, this application also provides a driving behavior semantic library construction system for autonomous driving, including:

[0278] The first acquisition module is used to acquire driving data samples of the vehicle at multiple historical time points. The driving data samples include trajectory samples, perception data samples and map data samples. The trajectory samples include the position, speed, acceleration and heading angle of multiple trajectory points.

[0279] The preprocessing module is used to preprocess driving data samples from multiple historical time points to obtain preprocessed driving data from multiple historical time points. The preprocessing includes timestamp alignment, outlier removal, coordinate system unification, and normalization.

[0280] The segmentation module is used to extract preprocessed trajectory data from the preprocessed driving data and segment the preprocessed trajectory data at multiple historical time points to obtain multiple trajectory segment samples.

[0281] The first word segmentation module is used to convert the trajectory fragment samples into a token sequence based on a pre-built trajectory word segmenter;

[0282] The structured processing module is used to extract the context features of the Token sequence from the preprocessed driving data, and construct a structured Token sequence based on the Token sequence and the context features of the Token sequence.

[0283] The semantic library construction module is used to perform semantic annotation on the structured token sequence to obtain a structured token sequence with semantic tags; and to construct an index structure for the structured token sequence with semantic tags to obtain a semantic library, wherein the index structure includes an inverted index, a vector index, and a classification index.

[0284] like Figure 4 As shown, this application also provides a contextualized retrieval method for a driving behavior semantic database for autonomous driving, including:

[0285] This application also provides a contextualized retrieval system for a semantic database of driving behavior for autonomous driving, including:

[0286] The second acquisition module is used to acquire the historical trajectory segment of the current vehicle for the first target time before the current time period, the predicted trajectory segment for the second target time after the current time point, and to acquire the perception data and map data of the current vehicle. The predicted trajectory segment is generated by the end-to-end model of the current vehicle.

[0287] The second word segmentation module is used to concatenate the historical trajectory fragment and the predicted trajectory fragment into the current scene trajectory fragment, and convert the current scene trajectory fragment into the current token sequence based on the pre-built trajectory word segmenter;

[0288] The feature extraction module is used to extract current context features from the current scene trajectory segment, the perception data, and the map data;

[0289] The preliminary retrieval module is used to perform a weighted voting-based retrieval in the semantic database based on the current token sequence and the current context features to obtain the semantic tags of the current token sequence;

[0290] The final retrieval module is used to perform a retrieval in the semantic database based on the current token sequence, the context features of the current token sequence, and the semantic tags of the current token sequence, and obtain retrieval results, wherein the retrieval results are used for online decision support and simulation test case generation.

[0291] This embodiment also provides an electronic terminal, including: a processor and a memory;

[0292] The memory is used to store computer programs, and the processor is used to execute the computer programs stored in the memory so that the terminal performs any of the methods in this embodiment.

[0293] As will be understood by those skilled in the art, the computer-readable storage medium described in this embodiment allows for the implementation of all or part of the steps in the above method embodiments by computer program-related hardware. The aforementioned computer program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0294] The electronic terminal provided in this embodiment includes a processor, a memory, a transceiver, and a communication interface. The memory and the communication interface are connected to the processor and the transceiver and complete communication between them. The memory is used to store computer programs, the communication interface is used to perform communication, and the processor and the transceiver are used to run the computer programs, so that the electronic terminal performs the steps of the above method.

[0295] In this embodiment, the memory may include random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage device.

[0296] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0297] In the above embodiments, although the present application has been described in conjunction with specific embodiments thereof, many substitutions, modifications, and variations of these embodiments will be apparent to those skilled in the art based on the foregoing description. The embodiments of the present application are intended to cover all such substitutions, modifications, and variations falling within the broad scope of the appended claims.

[0298] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this application should still be covered by the claims of this application.

Claims

1. A method for constructing a driving behavior semantic library for autonomous driving, characterized in that, Including the following steps: The vehicle's driving data samples at multiple historical time points are acquired. The driving data samples include trajectory samples, perception data samples, and map data samples. The trajectory samples include the position, speed, acceleration, and heading angle of multiple trajectory points. Driving data samples from multiple historical time points are preprocessed to obtain preprocessed driving data from multiple historical time points. The preprocessing includes timestamp alignment, outlier removal, coordinate system unification, and normalization. Preprocessed trajectory data is extracted from the preprocessed driving data, and the preprocessed trajectory data from multiple historical time points is segmented to obtain multiple trajectory segment samples. The trajectory segment samples are converted into token sequences based on a pre-built trajectory segmenter; The context features of the Token sequence are extracted from the preprocessed driving data, and a structured Token sequence is constructed based on the Token sequence and the context features of the Token sequence. The structured token sequence is semantically annotated to obtain a structured token sequence with semantic tags; and an index structure of the structured token sequence with semantic tags is constructed to obtain a semantic library, wherein the index structure includes an inverted index, a vector index, and a classification index.

2. The method for constructing a driving behavior semantic library for autonomous driving according to claim 1, characterized in that, Preprocessed trajectory data from multiple historical time points are segmented to obtain multiple trajectory segment samples, including: Preprocessed trajectory data from multiple historical time points are segmented using a pre-built sliding window to obtain multiple trajectory segment samples, which include historical trajectories and future trajectories of a target duration for the historical trajectories.

3. The method for constructing a driving behavior semantic library for autonomous driving according to claim 1, characterized in that, The trajectory segmenter includes an encoder for mapping trajectory segments to a latent space vector sequence, a codebook for converting the latent space vector sequence into a token sequence, and a decoder for mapping the token sequence to trajectory decoding segments. The method for constructing the trajectory segmenter includes: Obtain driving trajectory segment samples; The trajectory segment sample is input into the encoder to obtain the predicted token sequence output by the codebook and the trajectory decoded segment output by the decoder; The loss is calculated based on a pre-constructed loss function, the driving trajectory segment samples, the trajectory decoding segments, and the predicted token sequence, wherein the mathematical expression of the loss function is: In the formula, Indicates loss, This represents a sample of a driving trajectory segment. This represents a trajectory decoding segment. This represents the sequence of latent space vectors output by the encoder. This indicates that the gradient operation is stopped. For the codeword sequence in the codebook, The index is the token sequence. For loss balancing parameters; The trajectory segmenter is trained based on the loss.

4. The method for constructing a driving behavior semantic library for autonomous driving according to claim 3, characterized in that, Training the trajectory segmenter based on the loss includes: In the first stage, the codebook is frozen, and the encoder and decoder are trained based on the loss. At the end of the first stage, the second stage begins and the codebook is decoded. The encoder, the codebook, and the decoder are trained based on the loss until training is complete.

5. The method for constructing a driving behavior semantic library for autonomous driving according to claim 1, characterized in that, Extracting contextual features of the token sequence from the preprocessed driving data includes: The contextual features of the token sequence are obtained by extracting perception data, high-precision map data, and vehicle status data that are time-aligned with the token sequence from the preprocessed driving data.

6. The method for constructing a driving behavior semantic library for autonomous driving according to claim 1, characterized in that, Semantic annotation is performed on the structured token sequence to obtain a structured token sequence with semantic tags, including: The structured token sequences are clustered to obtain multiple clusters; and coarse classification labels are assigned to the structured token sequences within each cluster. Assign fine-classification labels to the structured token sequences within each cluster, and perform semantic annotation based on the coarse-classification labels and fine-classification labels of each structured token sequence to obtain structured token sequences with semantic labels.

7. A system for constructing a semantic library of driving behavior for autonomous driving, characterized in that, include: The first acquisition module is used to acquire driving data samples of the vehicle at multiple historical time points. The driving data samples include trajectory samples, perception data samples and map data samples. The trajectory samples include the position, speed, acceleration and heading angle of multiple trajectory points. The preprocessing module is used to preprocess driving data samples from multiple historical time points to obtain preprocessed driving data from multiple historical time points. The preprocessing includes timestamp alignment, outlier removal, coordinate system unification, and normalization. The segmentation module is used to extract preprocessed trajectory data from the preprocessed driving data and segment the preprocessed trajectory data at multiple historical time points to obtain multiple trajectory segment samples. The first word segmentation module is used to convert the trajectory fragment samples into a token sequence based on a pre-built trajectory word segmenter; The structured processing module is used to extract the context features of the Token sequence from the preprocessed driving data, and construct a structured Token sequence based on the Token sequence and the context features of the Token sequence. The semantic library construction module is used to perform semantic annotation on the structured token sequence to obtain a structured token sequence with semantic tags; and to construct an index structure for the structured token sequence with semantic tags to obtain a semantic library, wherein the index structure includes an inverted index, a vector index, and a classification index.

8. A scenario-based retrieval method for a semantic database of driving behavior for autonomous driving, characterized in that, include: The system acquires the historical trajectory segment of the current vehicle for the first target time period before the current time period, the predicted trajectory segment for the second target time period after the current time point, and the perception data and map data of the current vehicle. The predicted trajectory segment is generated by the end-to-end model of the current vehicle. The historical trajectory fragment and the predicted trajectory fragment are concatenated to form the current scene trajectory fragment, and the current scene trajectory fragment is converted into the current token sequence based on the pre-built trajectory segmenter; Extract current context features from the current scene trajectory segment, the perception data, and the map data; Based on the current token sequence and the current context features, a weighted voting-based retrieval is performed in the semantic database to obtain the semantic label of the current token sequence; Based on the current token sequence, the contextual features of the current token sequence, and the semantic tags of the current token sequence, a search is performed in the semantic database to obtain search results, wherein the search results are used for online decision support and simulation test case generation.

9. The scenario-based retrieval method for driving behavior semantic database for autonomous driving according to claim 8, characterized in that, Based on the current token sequence, the contextual features of the current token sequence, and the semantic tags of the current token sequence, a search is performed in the semantic database to obtain search results, including: Based on the inverted index in the semantic library, a first candidate set that locally matches the current token sequence is retrieved; based on the vector index in the semantic library, a second candidate set that matches the contextual features of the current token sequence is retrieved; and based on the classification index in the semantic library, a third candidate set that matches the semantic tags of the current token sequence is retrieved. Merge the first candidate set, the second candidate set, and the third candidate set to obtain the candidate set; Calculate the similarity between multiple structured token sequences in the candidate set and the current token sequence, wherein the similarity includes token sequence similarity, context feature similarity, and semantic label similarity; The K most similar structured token sequences are output as search results.

10. A contextualized retrieval system for a semantic database of driving behavior for autonomous driving, characterized in that: include: The second acquisition module is used to acquire the historical trajectory segment of the current vehicle for the first target time before the current time period, the predicted trajectory segment for the second target time after the current time point, and to acquire the perception data and map data of the current vehicle. The predicted trajectory segment is generated by the end-to-end model of the current vehicle. The second word segmentation module is used to concatenate the historical trajectory fragment and the predicted trajectory fragment into the current scene trajectory fragment, and convert the current scene trajectory fragment into the current token sequence based on the pre-built trajectory word segmenter; The feature extraction module is used to extract current context features from the current scene trajectory segment, the perception data, and the map data; The preliminary retrieval module is used to perform a weighted voting-based retrieval in the semantic database based on the current token sequence and the current context features to obtain the semantic tags of the current token sequence; The final retrieval module is used to perform a retrieval in the semantic database based on the current token sequence, the context features of the current token sequence, and the semantic tags of the current token sequence, and obtain retrieval results, wherein the retrieval results are used for online decision support and simulation test case generation.

Images