A trajectory discrete coding and serialization generation method and system for a visual language automatic driving model

Abstract

The present application relates to a kind of trajectory discrete coding and serialization generation method and system for visual language automatic driving model, belong to the field of automatic driving.The position increment between adjacent trajectory points is calculated, converted into distance component and angle component, and respectively performs compression and expansion mapping and non-uniform quantization, to obtain distance quantization interval identifier and angle quantization interval identifier, combined into discrete trajectory marker;Combined with vehicle motion state marker, form trajectory discrete coding sequence, output sequence containing structure marker and trajectory marker is output in unified autoregressive sequence generation interface by visual language automatic driving model;According to structure marker, trajectory answer section can be positioned and parsed to obtain trajectory marker subsequence, and the trajectory representation usable for downstream planning execution module is recovered by decoding.The present application considers trajectory representation accuracy, output form consistency and reversible recovery ability, and is suitable for the planning result generation and analysis of visual language automatic driving model.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and in particular to a method and system for trajectory discrete encoding and serialization generation for visual language autonomous driving models. Background Technology

[0002] In autonomous driving perception, prediction, planning, and control tasks, the future trajectory of a vehicle is typically represented as a series of continuous coordinate points. While this representation can directly reflect the shape of the trajectory, when used directly for sequence model training, generation, or transmission, it often requires the output of a large number of continuous values. This increases the burden of sequence modeling and makes the vehicle susceptible to the accumulation of numerical regression errors, thereby reducing the stability and consistency of trajectory generation.

[0003] In existing technologies, to make trajectory results suitable for discrete sequence modeling, there are schemes for discretizing and encoding the trajectory. For example, positions are bucketed according to Cartesian coordinates, or absolute coordinate encoding or uniform quantization encoding is performed on each trajectory point. Although the above methods can convert continuous trajectories into discrete markers to a certain extent, the following problems still exist: when the quantization interval is set finely, the size of the encoded markers increases and the sequence organization becomes more complex; when the quantization interval is set coarsely, key local changes such as close distances and small angles are difficult to accurately represent, thus affecting the accuracy of trajectory recovery.

[0004] On the other hand, in existing visual language autonomous driving solutions, the visual language model is usually responsible for environmental understanding and semantic reasoning, while trajectory planning results are still predicted through independently set action heads, dedicated trajectory output heads, or other additional output branches. Although this type of solution can obtain planning results, its planning output interface and language generation interface are separated, resulting in inconsistent model output formats and difficulty in simultaneously organizing structural information and planning results in the same output sequence.

[0005] Therefore, existing technologies still lack a comprehensive technical solution that can stably convert continuous trajectory point sequences into discrete trajectory markers suitable for sequence model processing, enable visual language autonomous driving models to output the trajectory marker sequence in a unified autoregressive sequence generation interface, and simultaneously locate, parse, and recover trajectory answer segments based on structural markers. Summary of the Invention

[0006] This invention addresses the technical problems existing in the background art by proposing a trajectory discrete encoding and serialization generation method and system for visual language autonomous driving models. It is used to convert continuous trajectory point sequences of autonomous driving into discrete trajectory marker sequences and output the trajectory marker sequences in a unified autoregressive sequence generation interface so as to locate, parse and recover the trajectory answer segments based on the structural markers.

[0007] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows:

[0008] A method for trajectory discrete encoding and serialization generation for visual language-based autonomous driving models includes the following steps:

[0009] Step S1: Receive multimodal input from the autonomous driving scenario. The multimodal input includes environmental visual information, language information related to the driving task, and vehicle historical motion state and / or current driving state information.

[0010] Step S2: Obtain a continuous trajectory point list of the autonomous driving trajectory, calculate the position increment between adjacent trajectory points, and convert the position increment into a trajectory increment in polar coordinates, wherein the trajectory increment includes a distance component and an angle component;

[0011] Step S3: Perform companding mapping on the distance component and the angle component respectively, and perform non-uniform quantization based on the values ​​after companding mapping to obtain the distance quantization interval identifier and the angle quantization interval identifier.

[0012] Step S4: Combine the distance quantization interval identifier and the angle quantization interval identifier into a discrete trajectory marker, generate a state marker corresponding to the vehicle's motion state, and form a discrete coding sequence from the state marker and the discrete trajectory marker.

[0013] Step S5: Input the multimodal input into the visual language autonomous driving model, so that the visual language autonomous driving model generates an output sequence containing structural markers and trajectory markers based on a unified autoregressive sequence generation interface;

[0014] Step S6: Determine the trajectory answer segment from the output sequence based on the structural markers, and parse the trajectory marker subsequence;

[0015] Step S7: Perform inverse quantization and inverse transformation from polar coordinates to Cartesian coordinates on the trajectory marker subsequence to obtain the trajectory representation, and output the trajectory representation to the downstream planning and execution module.

[0016] Furthermore, in step S3, the distance component and the angle component are quantized using different companding mapping parameters and different non-uniform quantization intervals, respectively.

[0017] In step S4, the state marker is used to characterize at least one of the following motion states of the vehicle: forward motion, reverse motion, or parking motion.

[0018] When the autonomous driving trajectory corresponds to a reversing state, the longitudinal component in the position increment is subjected to direction reversal processing, and the inverse recovery processing corresponding to the direction reversal processing is performed when the inverse transformation is used to restore the continuous trajectory.

[0019] When the quantization result corresponding to the distance quantization interval identifier is zero, the angle quantization interval identifier is set to a predetermined angle interval identifier to represent the invalid state of the angle component in the parking state.

[0020] In step S5, the output sequence also includes content markers corresponding to intermediate inferred content. These content markers are located outside the trajectory answer segment or in an output segment different from the trajectory answer segment.

[0021] In step S6, the structure markers include the start marker and end marker of the trajectory answer segment, and the trajectory marker subsequence is located between the start marker and the end marker and arranged in a predetermined time sequence.

[0022] A trajectory discrete encoding and serialization generation system for a visual language autonomous driving model is provided to implement the aforementioned trajectory discrete encoding and serialization generation method for a visual language autonomous driving model. The system includes a multimodal input processing module, an incremental conversion module, a companding quantization module, a discrete marker generation module, a unified sequence generation module, a trajectory segment parsing module, a trajectory decoding module, and a planning output module.

[0023] The multimodal input processing module is used to receive and organize environmental visual information, task language information, and vehicle status information in the autonomous driving scenario.

[0024] The incremental conversion module is used to convert a continuous trajectory point column into trajectory increments in polar coordinates.

[0025] The companding quantization module is used to obtain the distance quantization interval identifier and the angle quantization interval identifier;

[0026] The discrete marker generation module is used to generate discrete encoded sequences;

[0027] The unified sequence generation module is used to generate an output sequence containing structural markers and trajectory markers;

[0028] The trajectory segment parsing module is used to determine the trajectory answer segment based on the structural markers and parse the trajectory marker subsequence;

[0029] The trajectory decoding module is used to recover the trajectory marker subsequence into a trajectory representation;

[0030] The planning output module is used to output the trajectory representation to the downstream planning execution module.

[0031] Furthermore, in a preferred embodiment, the position increment, polar coordinate increment, compressive mapping result, quantization interval identifier, and discrete trajectory marker have the following correspondence: ;

[0032] Among them, (Δx) i , Δy i ) represents the position increment between adjacent trajectory points, (r i , θ i ) represents the polar coordinate increment, (r hat，i , θ hat，i ) represents the range component and angle component after companding mapping, (b r , b θ ) represents the distance quantization interval identifier and the angle quantization interval identifier, t i This represents the corresponding discrete trajectory marker. The correspondence described herein is used to illustrate preferred embodiments of the invention and does not constitute a sole limitation on companding mapping, quantization interval construction, or marker combination methods.

[0033] The present invention has the following advantages:

[0034] (1) The present invention adopts the polar coordinate representation of adjacent trajectory point increments, and performs companding mapping and non-uniform quantization on the distance component and the angle component respectively, so that the close-range and small-angle regions can obtain higher coding resolution, thereby improving the trajectory recovery accuracy.

[0035] (2) The present invention organizes the distance quantization interval identifier, the angle quantization interval identifier and the vehicle motion state marker into a discrete coding sequence, which can improve the consistency of trajectory representation under different motion states while controlling the sequence length and the scale of the marker.

[0036] (3) This invention organizes structure markers and trajectory markers through a unified autoregressive sequence generation interface, so that planning results can be generated in the same output sequence, and the trajectory answer segment can be located and parsed based on the structure marker, thereby improving the stability of trajectory result extraction.

[0037] (4) The present invention provides inverse quantization, coordinate recovery and modular system implementation methods corresponding to the discrete encoding and serialization generation process, so that the trajectory marker subsequence in the output sequence can be recovered into a trajectory representation that can be used by the downstream planning and execution module. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the embodiments of the present invention are briefly described below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0039] Figure 1 This is a schematic diagram of the overall process of the present invention;

[0040] Figure 2 This is a schematic diagram of the organizational structure of the output sequence in this invention;

[0041] Figure 3 This is a schematic diagram of the compressive amplification method and formula in this invention;

[0042] Figure 4 This is a system module block diagram of the present invention. Detailed Implementation

[0043] The technical solution of the present invention will be further described below with reference to the accompanying drawings and specific embodiments. It should be understood that the following embodiments are for illustrative purposes only and are not intended to limit the scope of protection of the present invention. Based on the embodiments of the present invention, equivalent substitutions or modifications made by those skilled in the art without creative effort should all fall within the scope of protection of the present invention.

[0044] This embodiment provides a method for trajectory discrete encoding and serialization generation for visual language-based autonomous driving models. The overall process is as follows: Figure 1 As shown, it includes the following steps:

[0045] Step S1: Receive multimodal input from the autonomous driving scenario. The multimodal input includes environmental visual information, language information related to the driving task, and vehicle historical motion state and / or current driving state information.

[0046] Step S2: Obtain the continuous trajectory point sequence of the autonomous driving trajectory, calculate the position increment between adjacent trajectory points, and convert the position increment into trajectory increment in polar coordinate form. Let the i-th trajectory point be (x... i , y i The (i+1)th trajectory point is (x i+1 , y i+1 If the position increment is 0, then the position increment can be expressed as: ;

[0047] Furthermore, the position increment is converted into a trajectory increment in polar coordinates to obtain the distance component. and angular components : , .

[0048] Step S3: Perform companding mapping on the distance component and the angle component respectively, and perform non-uniform quantization based on the values ​​after companding mapping to obtain the distance quantization interval identifier b. r and angle quantization interval identifier b θ The companding mapping and quantization relationship are as follows: Figure 3 As shown. In one implementation, the following companding function can be used for any companding quantity x to be compressed: ;

[0049] Correspondingly, the inverse composporation function can be expressed as: ;

[0050] Step S4: Combine the distance quantization interval identifier and the angle quantization interval identifier into a discrete trajectory marker, and combine this with the vehicle motion state to generate a state marker, forming a discrete coding sequence. In one implementation, (b r ,b θ The combined mapping is transformed into a single discrete trajectory label t. i ,Right now: ;

[0051] If the vehicle's motion state is denoted as s, then the discrete coded sequence can be represented as: ;

[0052] Step S5: Input the multimodal input into the visual language autonomous driving model, so that the visual language autonomous driving model generates an output sequence containing structural markers and trajectory markers based on a unified autoregressive sequence generation interface.

[0053] Step S6: Determine the trajectory answer segment from the output sequence based on the structural markers, and parse the trajectory marker subsequence.

[0054] Step S7: Perform inverse quantization and inverse transformation from polar coordinates to Cartesian coordinates on the trajectory marker subsequence to obtain the trajectory representation, and output the trajectory representation to the downstream planning and execution module. Let the distance component and angle component after inverse quantization be ri and θi, respectively, then the corresponding position increment can be recovered according to the following relationship: ;

[0055] Then, based on the trajectory starting point (x0, y0), the continuous trajectory point sequence can be recursively recovered: ;

[0056] This embodiment provides an output format and special scenario processing method applicable to the above method, and its schematic structure is as follows: Figure 2 As shown.

[0057] The output sequence includes at least a trajectory answer segment, which is defined by structural markers. In one specific implementation, the trajectory answer segment includes a trajectory answer start marker, multiple trajectory markers, and a trajectory answer end marker. The marker sequence between the trajectory answer start marker and the trajectory answer end marker can be determined as a trajectory marker subsequence.

[0058] Within the trajectory answer section, trajectory markers are arranged sequentially according to a predetermined time order. Each trajectory marker corresponds to a serialized trajectory representation unit in the planned trajectory, and multiple trajectory markers together represent the trajectory representation within the future planning time domain. In an optional implementation, the output sequence may further include content markers corresponding to intermediate inferred content.

[0059] For reversing scenarios, the longitudinal component of the position increment can be reversed. Let the original position increment be ( , If ), then the increment after flipping can be expressed as: ;

[0060] For parking scenarios, when the distance quantization interval is marked as b r The angle quantization interval is identified as b. θ When this is the case, the following constraint relationship can be adopted: ;

[0061] in, This identifies a predetermined angle range. The above processing enhances the consistency of trajectory sequence representations under different motion states.

[0062] This embodiment provides a trajectory discrete encoding and serialization generation system for visual language-based autonomous driving models, and its module structure is as follows: Figure 4 As shown, the system includes a multimodal input processing module, an incremental conversion module, a companding and quantization module, a discrete marker generation module, a unified sequence generation module, a trajectory segment parsing module, a trajectory decoding module, and a planning output module.

[0063] The multimodal input processing module is used to receive and organize environmental visual information, task language information, and vehicle status information in autonomous driving scenarios.

[0064] The incremental conversion module is used to calculate the position increment between adjacent trajectory points and convert the position increment into a polar coordinate trajectory increment that includes distance and angle components.

[0065] The companding quantization module is used to perform companding mapping and non-uniform quantization on the distance component and the angle component respectively, and output the distance quantization interval identifier and the angle quantization interval identifier.

[0066] The discrete marker generation module is used to combine the distance quantization interval identifier and the angle quantization interval identifier into a discrete trajectory marker, and combine them with the vehicle motion state to generate a state marker, so as to form a discrete coding sequence.

[0067] The unified sequence generation module is used to generate an output sequence containing structure markers and trajectory markers based on the unified autoregressive sequence generation interface. The organization method of the output sequence can be found in [reference needed]. Figure 2 .

[0068] The trajectory segment parsing module is used to determine the trajectory answer segment based on the structural markers in the output sequence, and to parse the trajectory marker subsequence.

[0069] The trajectory decoding module is used to perform inverse quantization and inverse transformation from polar coordinates to Cartesian coordinates on the trajectory marker subsequence to recover the trajectory representation. The relationship between the inverse quantization and inverse transformation can be found in [reference needed]. Figure 3 .

[0070] The planning output module is used to output the trajectory representation to the downstream planning execution module for use by the driving decision-making link.

Claims

1. A method for trajectory discrete encoding and serialization generation for visual language-based autonomous driving models, characterized in that, Includes the following steps: Step S1: Receive multimodal input from the autonomous driving scenario. The multimodal input includes environmental visual information, language information related to the driving task, and vehicle historical motion state and / or current driving state information. Step S2: Obtain a continuous trajectory point list of the autonomous driving trajectory, calculate the position increment between adjacent trajectory points, and convert the position increment into a trajectory increment in polar coordinates, wherein the trajectory increment includes a distance component and an angle component; Step S3: Perform companding mapping on the distance component and the angle component respectively, and perform non-uniform quantization based on the values ​​after companding mapping to obtain the distance quantization interval identifier and the angle quantization interval identifier. The distance component and the angle component are quantized using different companding mapping parameters and different non-uniform quantization intervals, respectively. Step S4: Combine the distance quantization interval identifier and the angle quantization interval identifier into a discrete trajectory marker, generate a state marker corresponding to the vehicle's motion state, and form a discrete coding sequence from the state marker and the discrete trajectory marker. Step S5: Input the multimodal input into the visual language autonomous driving model, so that the visual language autonomous driving model generates an output sequence containing structural markers and trajectory markers based on a unified autoregressive sequence generation interface; Step S6: Determine the trajectory answer segment from the output sequence based on the structural markers, and parse the trajectory marker subsequence; Step S7: Perform inverse quantization and inverse transformation from polar coordinates to Cartesian coordinates on the trajectory marker subsequence to obtain the trajectory representation, and output the trajectory representation to the downstream planning and execution module.

2. The trajectory discrete encoding and serialization generation method for a visual language-based autonomous driving model according to claim 1, characterized in that, In step S4, the state marker is used to characterize at least one of the following motion states of the vehicle: forward motion, reverse motion, or parking motion.

3. The trajectory discrete encoding and serialization generation method for a visual language-based autonomous driving model according to claim 1, characterized in that, When the autonomous driving trajectory corresponds to a reversing state, the longitudinal component in the position increment is subjected to direction reversal processing, and the inverse recovery processing corresponding to the direction reversal processing is performed when the inverse transformation is used to restore the continuous trajectory.

4. The trajectory discrete encoding and serialization generation method for a visual language-based autonomous driving model according to claim 1, characterized in that, When the quantization result corresponding to the distance quantization interval identifier is zero, the angle quantization interval identifier is set to a predetermined angle interval identifier to represent the invalid state of the angle component in the parking state.

5. The trajectory discrete encoding and serialization generation method for a visual language-based autonomous driving model according to claim 1, characterized in that, In step S5, the output sequence also includes content markers corresponding to intermediate inferred content. These content markers are located outside the trajectory answer segment or in an output segment different from the trajectory answer segment.

6. The trajectory discrete encoding and serialization generation method for a visual language-based autonomous driving model according to claim 1, characterized in that, In step S6, the structure markers include the start marker and end marker of the trajectory answer segment, and the trajectory marker subsequence is located between the start marker and the end marker and arranged in a predetermined time sequence.

7. A trajectory discrete encoding and serialization generation system for a visual language-based autonomous driving model, used to implement the trajectory discrete encoding and serialization generation method for a visual language-based autonomous driving model as described in any one of claims 1 to 6, characterized in that, It includes a multimodal input processing module, an incremental conversion module, a companding and quantization module, a discrete marker generation module, a unified sequence generation module, a trajectory segment parsing module, a trajectory decoding module, and a planning output module; The multimodal input processing module is used to receive and organize environmental visual information, task language information, and vehicle status information in the autonomous driving scenario. The incremental conversion module is used to convert a continuous trajectory point column into trajectory increments in polar coordinates. The companding quantization module is used to obtain the distance quantization interval identifier and the angle quantization interval identifier; The discrete marker generation module is used to generate discrete encoded sequences; The unified sequence generation module is used to generate an output sequence containing structural markers and trajectory markers; The trajectory segment parsing module is used to determine the trajectory answer segment based on the structural markers and parse the trajectory marker subsequence; The trajectory decoding module is used to recover the trajectory marker subsequence into a trajectory representation; The planning output module is used to output the trajectory representation to the downstream planning execution module.

Images