VLA end-to-end autonomous driving safety navigation method and system based on kinetic boundary constraint

Abstract

The present application relates to a VLA end-to-end autonomous driving safety navigation method and system based on kinetic boundary constraint, including obtaining images, navigation instructions, driving history and navigation state, using visual semantics and time alignment inertia response residual to obtain road adhesion posterior; VLA represents actions as spatial path and longitudinal motion sequence, link embedding differentiable kinetic alignment layer, constructing prediction time domain dynamic feasible set according to adhesion posterior, path curvature, longitudinal velocity and longitudinal acceleration. The alignment layer constrains fine-tuning through physical loss guidance, verifies candidate trajectories combined with physical state to dynamically mask or risk penalty constraint inaction, executes safety degradation when infeasible, and outputs safe trajectories or control instructions conforming to kinetic boundary. The present application introduces vehicle physical state, road adhesion change and prediction time domain kinetic constraint into VLA action generation, improves the dynamic feasibility, continuity and stability of navigation actions under low adhesion, high curvature and high dynamics.

Description

Technical Field

[0001] This invention relates to the field of intelligent connected vehicle and autonomous driving chassis control technology, and in particular to a VLA end-to-end autonomous driving safety navigation method and system based on dynamic boundary constraints. Background Technology

[0002] With the rapid development of intelligent connected vehicle technology, high-level autonomous driving has become a core direction of transformation in the automotive industry. In recent years, end-to-end autonomous driving technology, characterized by the integration of perception, decision-making, and control, has demonstrated powerful data fitting and scenario generalization capabilities. In particular, with the introduction of embodied intelligence models such as Vision-Language-Action (VLA), autonomous vehicles have gained unprecedented environmental semantic cognition and complex interaction capabilities, propelling autonomous driving from traditional rule-driven to a new stage of data-driven and generative cognition.

[0003] However, ensuring the safe operation of end-to-end systems is a crucial prerequisite for the large-scale deployment of high-level autonomous driving. Existing end-to-end large-model navigation solutions are mostly driven by massive amounts of purely visual data, generally exhibiting a weakness in prioritizing semantic understanding over physical constraints. Because the autoregressive generation mechanism of embodied intelligent large models is inherently probabilistic and random, its decision-making commands are often based solely on the statistical distribution patterns of training data, severely lacking real-time quantitative understanding of the vehicle's current objective physical limits. This lack of physical common sense makes autonomous driving systems highly susceptible to physical illusions when facing complex, long-tailed conditions such as low adhesion, high curvature, and emergency obstacle avoidance, easily outputting dangerous commands that appear geometrically reasonable but are dynamically unenforceable. For vehicles operating at high speeds in open, dynamic environments, any violation of physical laws can pose a serious safety risk.

[0004] Currently, safety alignment and constraint mechanisms for end-to-end autonomous driving mostly remain at the level of macroscopic semantic rule constraints or simple post-processing trajectory filtering. Chinese invention publication CN122058956A discloses a unified intelligent chassis control method based on a vision-language-action model and dynamic feasible domain projection. This invention introduces intermediate action descriptions and dynamic feasible domain projection mechanisms between high-level semantic decision-making (VLA) and low-level intelligent chassis control, improving the feasibility, safety, stability, interpretability, and multi-actuator collaborative control capabilities of autonomous vehicles in complex scenarios. However, these output-side projection or filtering schemes have two significant engineering limitations: First, they lack deep coupling to the underlying physical state, failing to achieve real-time and accurate quantification of the vehicle's highly nonlinear dynamic envelope and ignoring transient changes in tire nonlinear mechanical characteristics and road adhesion conditions. Second, as an external post-processing filtering module, there is an inherent conflict between its rigid physical safety baseline and the soft generative reasoning logic of the large model. The post-processing module not only suffers from response lag but also struggles to eradicate the generation of dangerous action commands at the underlying model reasoning level. If a large model continuously outputs unsafe commands that exceed the dynamic limits, simple interception and filtering can easily cause the underlying actuators to fail to receive continuous and smooth control signals, which can lead to the system falling into control deadlock or decision oscillation, and the dynamic stability of the vehicle can still not be fully guaranteed.

[0005] In summary, there is currently a lack of a VLA end-to-end autonomous driving safety navigation method and system based on dynamic boundary constraints, which can introduce vehicle dynamic safety operating domain, road surface adhesion posterior, and sequence-level constrained decoding into the action generation process while maintaining VLA semantic reasoning capabilities, in order to solve or partially solve the above problems. Summary of the Invention

[0006] The purpose of this invention is to overcome the defects of the prior art by providing a VLA end-to-end autonomous driving safety navigation method and system based on dynamic boundary constraints. This aims to solve or partially solve the control discontinuity problem that may be caused by traditional external post-processing filtering, as well as the problems that are difficult to quantify in real time due to the lack of physical knowledge, and the difficulty in incorporating the nonlinear mechanical characteristics of tires and transient changes in road adhesion conditions into the motion generation process.

[0007] The objective of this invention can be achieved through the following technical solutions: According to one aspect of the present invention, a VLA end-to-end automated driving safety navigation method based on dynamic boundary constraints is provided, the method comprising: S1. Acquire multi-source data, including historical perception feature sequences output by the vehicle perception system, current environmental images, navigation commands, and real-time physical state data output by the vehicle chassis and integrated navigation system; S2. Input the multi-source data into a pre-trained lightweight dynamic envelope regression network, and combine the vehicle's real-time physical state, visual attachment prior, and inertial response residual correction results to end-to-end fit and output the nonlinear dynamic safety operating domain of the vehicle at the current moment. S3. The nonlinear dynamic safety operation domain is transformed into a structured dynamic constraint code, which is input into the vision-language-action large model along with the current environment image. The probability distribution of the initial navigation action sequence is generated by physical constraint reasoning, and candidate trajectories corresponding to candidate action tokens or candidate action token prefixes are generated. S4. In each autoregressive decoding step of the vision-language-action big model, the candidate trajectory is input into the dynamic alignment layer; the dynamic alignment layer takes the current integrated navigation state as the initial state, recursively calculates the vehicle response of the candidate trajectory in the prediction time domain, and constructs a dynamic mask matrix or risk penalty term based on the relationship between the candidate trajectory and the nonlinear dynamic safety operation domain to correct the generation probability of dangerous action tokens that exceed the vehicle safety dynamic boundary. S5. Based on the corrected action probability, perform constraint decoding and output dynamic feasible actions, safe trajectories, or control commands.

[0008] The nonlinear dynamic safety operating domain includes a dynamic boundary parameter vector and a predicted temporal dynamic feasible set. The lightweight dynamic envelope regression network is used to obtain the dynamic boundary parameters based on historical perception features, integrated navigation state, and the posterior of the road surface adhesion coefficient; the predicted temporal dynamic feasible set is used to verify candidate action tokens or candidate trajectories corresponding to candidate action token prefixes. Structured dynamic constraint encoding can employ at least one of structured natural language cue encoding, numerical constraint vectors, or dynamic constraint embedding features.

[0009] The dynamic boundary estimation module can be trained using simulation data, real vehicle calibration data, or samples calculated offline by the vehicle dynamics model. Training samples include historical perception features, integrated navigation states, visual adhesion priors, road adhesion coefficient posteriors, and vehicle execution inputs. Supervision labels include at least one of the following: upper limit of safe speed, upper limit of curvature, upper limit of longitudinal acceleration, upper limit of lateral acceleration, and stability margin. The training loss employs squared error, Huber loss, or regression loss with physical consistency constraints to ensure that the output dynamic boundary parameters remain consistent with the velocity-curvature-adhesion coupling relationship.

[0010] This invention establishes a training-inference consistent dynamic constraint mechanism by embedding a differentiable dynamic alignment layer into the VLA action generation chain. The alignment layer calculates the physical constraint loss based on the road surface adhesion posterior, the integrated navigation state, and the predicted temporal dynamic feasible set, and propagates the physical gradient to the candidate path generation, longitudinal motion generation, and semantic-physical estimation components, internalizing the velocity-curvature-adhesion coupling relationship into the VLA action distribution. After fine-tuning the dynamic constraints, the alignment layer continues to evaluate the VLA candidate trajectory in conjunction with the real-time physical state and constrains inactive actions through dynamic masking or risk penalties, thereby avoiding the degradation of physical constraints into independent output-side post-processing.

[0011] As a preferred technical solution, the nonlinear dynamic safe operating domain can be jointly described by the dynamic boundary parameter vector and the predicted time-domain dynamic feasible set, wherein the dynamic boundary parameter vector The mathematical expression is: in, It is a nonlinear mapping function. These are the network weight parameters. For a moment The acquired historical sensory feature sequence, For real-time physical state data, This represents the maximum safe curvature allowed for the vehicle under current operating conditions. This is the limiting lateral acceleration. This is the posterior estimate of the road surface adhesion coefficient. The upper limit of safe speed is related to curvature and adhesion conditions. This is the matrix transpose.

[0012] As a preferred technical solution, the real-time physical state data includes longitudinal velocity, lateral velocity, yaw rate, longitudinal acceleration, and lateral acceleration in the vehicle coordinate system. in, For longitudinal velocity, For lateral velocity, The yaw rate is angular velocity. For longitudinal acceleration, This is lateral acceleration.

[0013] As a preferred technical solution, the posterior of the road surface adhesion coefficient is determined jointly by visual semantic prior and inertial response residual. The visual semantic module outputs the adhesion prior based on semantics such as road surface material, wetness, water accumulation, snow accumulation, or icing. in, Let the road surface adhesion coefficient at the current moment be a prior random variable. The prior mean of the road surface adhesion coefficient estimated by the visual semantic module. To estimate the uncertainty or standard deviation of visual attachment, This indicates a normal distribution.

[0014] Using the actual longitudinal motion demand and steering input from the previous cycle, the vehicle acceleration response at the current moment is predicted through a vehicle dynamics model: in, The vehicle's acceleration response at the current moment is predicted by the vehicle dynamics model. For vehicle dynamic response prediction function, The steering, drive, or braking inputs actually executed in the previous cycle. The road surface adhesion coefficient at the current moment is a priori. These are the parameters for the vehicle dynamics model.

[0015] The inertial response residual is constructed based on the measured vehicle acceleration and predicted acceleration response output by the integrated navigation system: in, The measured vehicle acceleration is output by the integrated navigation system. For inertial response residuals, This refers to the vehicle acceleration response predicted by the vehicle dynamics model.

[0016] The visual adhesion prior is corrected using the inertial response residual to obtain the posterior road adhesion coefficient: in, The correction gain is determined based on visual uncertainty and inertial observation noise. For the amplitude limiting function, and These are the preset lower and upper limits of the road surface adhesion coefficient, respectively.

[0017] As a preferred technical solution, candidate actions Action tokens can be used to represent these actions, and they can also be further mapped to spatial paths and longitudinal motion sequences. in, Used to describe the vehicle's future spatial path. This describes the changes in vehicle velocity and longitudinal acceleration along the path. The lateral acceleration induced by path curvature approximately satisfies... , and These are longitudinal velocity and longitudinal acceleration, respectively. For the first Path curvature at each predicted location To predict the number of discrete locations in the time domain, For the first Path points, For the first The arc length position at the predicted location.

[0018] As a preferred technical solution, the compliance action token space is not determined solely by a single token, but rather by mapping candidate token prefixes to predicted time-domain candidate trajectories and verifying them point-by-point based on the predicted time-domain dynamic feasible set. The predicted time-domain dynamic feasible set can be represented as: in, For spatial path, This is a longitudinal motion sequence generated along the spatial path. For the first Path curvature at each predicted location and These are the corresponding longitudinal velocity and longitudinal acceleration, respectively. It is the acceleration due to gravity. This is used to predict the number of discrete locations in the time domain. The formula reflects the coupling relationship between velocity, curvature, longitudinal acceleration, and road surface adhesion.

[0019] As a preferred technical solution, the process of correcting the generation probability of dangerous action tokens that exceed the vehicle safety dynamics boundary includes: The action decoding word table of the visual-language-action big model is denoted as follows: In the first At each decoding position, the existing candidate action token prefix is ​​combined with the current candidate action token to form a... And mapped to candidate trajectories through a preset action dictionary, trajectory decoder, or path-velocity generator. : in, This is a mapping function from candidate action token prefixes to candidate trajectories. This represents the current combined navigation state.

[0020] Using the current integrated navigation state as the initial state, the vehicle dynamics model is used to recursively deduce the vehicle response of the candidate trajectory in the prediction time domain, and point-by-point determination is made as to whether the candidate trajectory belongs to the dynamically feasible set. The dynamic masking function is expressed as: When the degree of dynamic boundary violation of the candidate trajectory is less than a preset threshold, the dynamic mask function is replaced with a finite negative risk penalty term determined according to the degree of dynamic boundary violation; when all candidate action tokens are infeasible, at least one of the following safety degradation strategies is triggered: controlled deceleration, maintaining the feasible trajectory of the previous cycle, or emergency braking trajectory.

[0021] After applying a dynamic masking function or risk penalty term, the probability distribution of secure decoding actions is obtained by activating the Softmax function: in, When the first candidate action token is a given prefix The probability of a safe action for each candidate action token. For the size of the action decoding vocabulary, , The first The and the first The original logarithmic probability of each candidate action token. , These are the dynamic masking functions or risk penalty items for the corresponding candidate action tokens; and the outputs of security token sequences, security trajectories, or control commands are achieved through greedy sampling, bundle search, or restricted sampling.

[0022] As a preferred technical solution, the vision-language-action large model is obtained through LoRA fine-tuning, adapter fine-tuning, cue optimization, or full-parameter fine-tuning. During training, one or more of the following can be added to the model training objective: dynamic feasibility loss, inertial consistency loss, action correction bias loss, and trajectory smoothing loss, so that both the training and inference phases are subject to dynamic boundary constraints. The total loss can be expressed as: in, To monitor learning loss, For dynamic feasibility loss, For inertial consistency loss, To correct for deviation losses in action, For trajectory smoothing loss, , , and These are the weighting coefficients.

[0023] The differentiable dynamic alignment layer serves as a unified physical interface connecting VLA semantic action generation and vehicle dynamics boundaries, receiving candidate paths, longitudinal motion sequences, posterior road adhesion coefficients, and integrated navigation states. This alignment layer propagates dynamic feasibility errors to the VLA parameters through the following physical constraint loss, enabling the model to progressively learn a velocity-curvature-adhesion consistent candidate action distribution. After fine-tuning, the same alignment layer continues to be used for online dynamic feasibility evaluation, generating dynamic masks or risk penalties based on real-time vehicle states.

[0024] The dynamic feasibility loss is used to penalize candidate trajectories for violating the velocity-curvature-adhesion coupling boundary in the prediction time domain. Inertial consistency loss is used to ensure that the posterior dynamic prediction response corresponding to the road adhesion coefficient remains consistent with the measured response of the integrated navigation system. in, For inertial consistency error, The measured vehicle acceleration output by the integrated navigation system. For vehicle dynamic response prediction function, This is the inertial observation noise covariance matrix or weight matrix.

[0025] Motion correction bias loss is used to limit the deviation between the original longitudinal motion sequence and the feasible correction result: in, This refers to the longitudinal motion sequence after dynamic feasible set projection, risk penalty correction, or safety reconstruction. The original longitudinal motion sequence generated for the VLA large model. The weight matrix for correcting deviations in the action.

[0026] Trajectory smoothing loss is used to suppress abrupt changes in longitudinal acceleration and path curvature at adjacent predicted locations: The time-domain dynamic feasible set verification process simultaneously participates in VLA parameter optimization and online candidate trajectory evaluation, enabling the model to learn to reduce the degree of dynamic boundary violation and maintain consistency between the optimization objective and online constraints.

[0027] According to another aspect of the present invention, a VLA end-to-end automated driving safety navigation system based on dynamic boundary constraints is provided, the system being used to execute the VLA end-to-end automated driving safety navigation method based on dynamic boundary constraints as described above, the system comprising: The perception and status acquisition module is used to synchronously acquire the historical perception feature sequence of the vehicle perception system, the current environmental image, navigation commands, and the real-time physical status data output by the vehicle chassis and integrated navigation system. The dynamic envelope and adhesion estimation module is used to output the nonlinear dynamic safety operating domain using a lightweight dynamic envelope regression network, and to obtain the posterior of the road adhesion coefficient by combining the inertial response residual. The VLA candidate action generation module is used to receive the current environment image and structured dynamics constraint encoding, and output the original action score of the candidate action token and its corresponding candidate trajectory. The dynamic alignment module is used to generate physical constraint loss based on the deviation between candidate trajectories and the predicted temporal dynamic feasible set during the training phase and backpropagate it to the vision-language-action large model. During the inference phase, it performs sequence-level dynamic verification of candidate trajectories based on the current integrated navigation state, the posterior of the road surface adhesion coefficient, and the predicted temporal dynamic feasible set, and generates dynamic masks or risk penalties. The dynamic mask restricted decoding module is used to construct a dynamic mask matrix or risk penalty item based on the verification results, update the probability of candidate actions, and perform security degradation processing when there are no actions, outputting a compliant security trajectory or control instructions.

[0028] Compared with the prior art, the present invention has at least one of the following beneficial effects: (1) In the autoregressive generation stage of the vision-language-action model, the present invention verifies the predicted temporal trajectory corresponding to the candidate action token prefix, and constructs a dynamic mask or risk penalty based on the verification results, thereby reducing the repeated infeasible output and trajectory discontinuity caused by hard filtering after the complete trajectory is generated, and improving the dynamic compliance and vehicle executability of the generated action.

[0029] (2) This invention uses a lightweight dynamic envelope regression network to end-to-end fit and output the nonlinear dynamic safety operating domain of the vehicle at the current moment. It also combines the visual attachment prior and the inertial response residual to obtain the road surface adhesion coefficient posterior, thereby realizing the collaborative modeling of visual semantic information, vehicle motion state and dynamic boundary information. This solves the problem that pure semantic action generation is difficult to perceive the vehicle's physical executable boundary in real time, thereby reducing the output of dynamic non-actions and improving the safety, stability and vehicle executability of navigation commands.

[0030] (3) This invention uses the VLA large model to generate the probability distribution of the initial navigation action sequence, and through the structured dynamics constraint coding and dynamic mask restricted decoding mechanism, it corrects the action probability of candidate actions that exceed the vehicle dynamics safety boundary, thereby outputting safe navigation instructions that are more in line with the current vehicle dynamics constraints, and enhancing the transparency, continuity and traceability of the end-to-end autonomous driving system in safety-critical tasks. Attached Figure Description

[0031] Figure 1 A general flowchart of a VLA end-to-end autonomous driving safety navigation method based on dynamic boundary constraints provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the VLA action decoding and correction process based on dynamic constraint input in an embodiment of the present invention; Figure 3 This is a structural block diagram of a VLA end-to-end autonomous driving safety navigation system based on dynamic boundary constraints, provided for an embodiment of the present invention. Detailed Implementation

[0032] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0033] Example 1 To address the problems existing in the aforementioned prior art, this embodiment provides a VLA end-to-end autonomous driving safety navigation method based on dynamic boundary constraints. This method is mainly executed by the onboard computing platform, such as... Figure 1 As shown, the specific steps include: S1. Acquire multi-source data, which includes historical perception feature sequences output by the vehicle perception system, current environmental images, navigation commands, and real-time physical state data output by the vehicle chassis and integrated navigation system.

[0034] The onboard computing platform synchronously acquires multi-source data through sensor interfaces, including historical perception feature sequences extracted from the forward-facing camera and LiDAR. And real-time physical state data acquired through the drive-by-wire chassis and inertial navigation unit. .in, The vehicle's current longitudinal speed, For lateral velocity, The yaw rate is angular velocity. For longitudinal acceleration, This is lateral acceleration.

[0035] The vehicle dynamics predictor can use a planar three-degree-of-freedom single-track model, whose longitudinal, lateral, and yaw dynamics are expressed as follows: in, For the overall vehicle quality, For the moment of inertia of yaw rotation, and These are the distances from the center of mass to the front and rear axles, respectively. For the front wheel steering angle, , , and These are the longitudinal and lateral tire forces on the front and rear axles, respectively. The rate of change of longitudinal velocity, The yaw rate is angular velocity. The lateral velocity in the vehicle coordinate system. The rate of change of lateral velocity, The longitudinal velocity is in the vehicle coordinate system.

[0036] The longitudinal acceleration in the candidate trajectory represents the longitudinal motion demand, and the path curvature is converted into an equivalent steering input based on the monorail geometry; the vehicle dynamics predictor is used to obtain the longitudinal, lateral, and yaw responses of the candidate trajectory.

[0037] S2. Input multi-source data into a pre-trained lightweight dynamic envelope regression network, and combine the vehicle's real-time physical state, visual attachment prior, and inertial response residual correction results to end-to-end fit and output the nonlinear dynamic safe operating domain of the vehicle at the current moment.

[0038] The system constructs a lightweight multilayer perceptron deep regression network. The historical perception feature sequence, integrated navigation state, and attached posterior are concatenated and input into this network. An end-to-end fitting output is then performed to determine the dynamic boundary parameter vector at the current moment. Simultaneously, the system constructs a predictive time-domain dynamic feasible set based on the dynamic boundary parameter vector. This is used for sequence-level verification of subsequent candidate action tokens.

[0039] For example, at a specific moment, a vehicle is entering a curve with dark ice on its surface at a high speed. The visual semantic module provides a low prior estimate of adhesion based on the road surface appearance; subsequently, the system predicts the vehicle response using a vehicle dynamics model based on the steering and braking inputs executed in the previous cycle, and compares it with the measured acceleration output by the integrated navigation system. If the measured response indicates that the vehicle's actual stability margin is less than the visual prior estimate, the correction module reduces the adhesion coefficient to obtain the posterior road surface adhesion, and accordingly reduces the allowable safe curvature and speed limit.

[0040] The nonlinear dynamic safe operating domain includes a dynamic boundary parameter vector and a predicted time-domain dynamic feasible set constructed from the dynamic boundary parameter vector. The mathematical expression is: in, It is a nonlinear mapping function. These are the network weight parameters. For a moment The acquired historical sensory feature sequence, For real-time physical state data, This represents the maximum safe curvature allowed for the vehicle under current operating conditions. This is the limiting lateral acceleration. This is the posterior estimate of the road surface adhesion coefficient. To ensure a safe speed limit in accordance with curvature and adhesion conditions, This is the matrix transpose.

[0041] Real-time physical state data includes integrated navigation state and vehicle execution input. The integrated navigation state is represented as follows: in, For longitudinal velocity, For lateral velocity, The yaw rate is angular velocity. For longitudinal acceleration, This is lateral acceleration.

[0042] The process of obtaining the posterior estimate of the road surface adhesion coefficient includes: obtaining the uncertainty of the prior and visual adhesion estimates of the road surface adhesion coefficient based on the semantics of road surface material, wetness, water accumulation, snow accumulation, or icing in the current environmental image; obtaining the predicted acceleration response using the vehicle execution input and vehicle dynamics model of the previous cycle; calculating the inertial response residual between the measured acceleration and the predicted acceleration response of the integrated navigation system, and using the inertial response residual to correct the prior of the road surface adhesion coefficient to obtain the posterior of the road surface adhesion coefficient, and finally obtaining the posterior estimate of the road surface adhesion coefficient.

[0043] The visual semantic module attaches priors based on semantic outputs such as road surface material, wetness, water accumulation, snow accumulation, or ice formation. in, Let the road surface adhesion coefficient at the current moment be a prior random variable. The prior mean of the road surface adhesion coefficient estimated by the visual semantic module. To estimate the uncertainty or standard deviation of visual attachment, This indicates a normal distribution.

[0044] Using the actual longitudinal motion demand and steering input from the previous cycle, the vehicle acceleration response at the current moment is predicted through a vehicle dynamics model: in, The vehicle's acceleration response at the current moment is predicted by the vehicle dynamics model. For vehicle dynamic response prediction function, The steering, drive, or braking inputs actually executed in the previous cycle. The road surface adhesion coefficient at the current moment is a priori. These are the parameters for the vehicle dynamics model.

[0045] The inertial response residual is constructed based on the measured vehicle acceleration and predicted acceleration response output by the integrated navigation system: in, The measured vehicle acceleration is output by the integrated navigation system. For inertial response residuals, This refers to the vehicle acceleration response predicted by the vehicle dynamics model.

[0046] By correcting the visual adhesion prior using the inertial response residuals, the posterior road adhesion coefficient is obtained: in, The correction gain is determined based on visual uncertainty and inertial observation noise. For the amplitude limiting function, and These are the preset lower and upper limits of the road surface adhesion coefficient, respectively.

[0047] S3. Transform the nonlinear dynamic safety operation domain into a structured natural language prompt code, and input it along with the current environment image into the vision-language-action large model. Use physical constraint reasoning to generate the probability distribution of the initial navigation action sequence, and generate candidate trajectories corresponding to candidate action tokens or candidate action token prefixes.

[0048] The low-dimensional physical boundary index obtained above This can be converted into structured natural language prompt encoding or numerical embedding. For example, it could be converted to: [Physics Constraint]: Current estimated roadfriction is low. Maximum safe lateral acceleration is limited. Avoid hardsteering and reduce speed before entering the curve. Subsequently, the structured prompt, numerical embedding, or a combination of both, is input along with the current environmental image into a finely tuned vision-language-action model. The model can then generate an initial action token probability distribution according to stages such as environmental perception, physical state cognition, dynamic safety boundary reasoning, and navigation action space decision-making.

[0049] The vision-language-action model employs joint fine-tuning with dynamic constraints. The candidate paths and longitudinal motion sequences generated by the model are processed by a differentiable dynamic alignment layer to calculate dynamic feasibility loss, inertial consistency loss, action correction deviation loss, and trajectory smoothing loss, enabling physical gradients to act on model parameters. After fine-tuning, the alignment layer remains embedded in the online action generation link and combines real-time integrated navigation status and road surface adhesion coefficient posterior evaluation to assess the dynamic feasibility of candidate trajectories.

[0050] Candidate Action Action tokens can be used to represent these actions, or they can be further mapped to spatial paths and longitudinal motion sequences, as follows: in, Used to describe the vehicle's future spatial path. This describes the changes in vehicle velocity and longitudinal acceleration along a path. The lateral acceleration induced by path curvature approximately satisfies... , and These are longitudinal velocity and longitudinal acceleration, respectively. For the first Path curvature at each predicted location To predict the number of discrete locations in the time domain, For the first Path points, For the first The arc length position at the predicted location.

[0051] The compliance action token space is not determined solely by a single token, but rather by mapping candidate token prefixes to predicted time-domain candidate trajectories and verifying them point-by-point based on the predicted time-domain dynamic feasible set. The predicted time-domain dynamic feasible set can be represented as: in, For spatial path, This is a longitudinal motion sequence generated along a spatial path. For the first Path curvature at each predicted location and These are the corresponding longitudinal velocity and longitudinal acceleration, respectively. It is the acceleration due to gravity. This is used to predict the number of discrete locations in the time domain. The formula reflects the coupling relationship between velocity, curvature, longitudinal acceleration, and road surface adhesion.

[0052] S4. In each autoregressive decoding step of the vision-language-action model, the candidate action token or candidate action token prefix is ​​mapped to the candidate trajectory, and the candidate trajectory is input into the dynamic alignment layer. The dynamic alignment layer recursively calculates the vehicle response of the candidate trajectory in the prediction time domain based on the current integrated navigation state as the initial state, and constructs a dynamic mask matrix or risk penalty term based on the relationship between the candidate trajectory and the nonlinear dynamic safety operation domain to correct the generation probability of dangerous action tokens that exceed the vehicle safety dynamic boundary.

[0053] The action decoding word table of the visual-language-action model is denoted as follows: In the first At each decoding position, the existing candidate action token prefix is ​​combined with the current candidate action token to form a... And mapped to candidate trajectories through a preset action dictionary, trajectory decoder, or path-velocity generator. : in, This is a mapping function from candidate action token prefixes to candidate trajectories. This represents the current combined navigation state.

[0054] Combination Figure 2 As shown, the action probability correction mechanism includes VLA raw decoding output, candidate trajectory mapping, dynamic alignment and feasibility verification, action probability correction, dynamic feasible action output, and dynamic constraint input. Specifically, in the current decoding step of the vision-language-action model, candidate action token prefixes and their original action scores are first generated from the VLA raw decoding output. Subsequently, candidate trajectory mapping maps the candidate action token prefixes to predicted temporal candidate trajectories through a preset action dictionary, trajectory decoder, or path-velocity generator. Dynamic alignment and feasibility verification receives candidate trajectories and dynamic constraint inputs, including integrated navigation state, road surface adhesion posterior, and vehicle dynamics model, used to recursively deduce the vehicle response of candidate trajectories in the predicted temporal domain and determine whether candidate trajectories meet the dynamic feasible set constraints in the predicted temporal domain. Based on the verification results, action probability correction generates dynamic masks or risk penalty terms to correct the original action scores of candidate action tokens that exceed the vehicle safety dynamics boundary. Finally, dynamic feasible action outputs a safety token sequence, a safety trajectory, or a control command.

[0055] In the action probability correction, the system maps the nonlinear dynamic safe operating domain to a large model action decoding vocabulary based on the verification results of dynamic alignment and feasibility verification. Compliance Action Subspace For example, in icy cornering scenarios, action tokens for emergency turns, rapid acceleration, or high-speed cornering can be determined to be outside the current prediction time-domain dynamic feasible set and excluded. In addition, it may be subject to a limited negative risk penalty.

[0056] For vocabulary The first in Action Token The function to apply the dynamic mask is: Let the original unnormalized logarithmic value of the action token be... After applying a dynamic masking function or a risk penalty term, the probability distribution of secure actions after restricted decoding is obtained: in, When the first candidate action token is a given prefix The probability of a safe action for each candidate action token. For the size of the action decoding vocabulary, , The first The and the first The original logarithmic probability of each candidate action token. , These are the dynamic masking functions or risk penalty items for the corresponding candidate action tokens; and the outputs of security token sequences, security trajectories, or control commands are achieved through greedy sampling, bundle search, or restricted sampling.

[0057] The vision-language-action large model is obtained through LoRA fine-tuning, adapter fine-tuning, cue optimization, or full-parameter fine-tuning. During training, one or more of the following can be added to the model training objective: dynamic feasibility loss, inertial consistency loss, action correction bias loss, and trajectory smoothing loss, so that both the training and inference phases are subject to dynamic boundary constraints. The total loss can be expressed as: in, To monitor learning loss, For dynamic feasibility loss, For inertial consistency loss, To correct for deviation losses in action, For trajectory smoothing loss, , , and These are the weighting coefficients.

[0058] The differentiable dynamic alignment layer serves as a unified physical interface connecting VLA semantic action generation and vehicle dynamics boundaries, receiving candidate paths, longitudinal motion sequences, posterior road adhesion coefficients, and integrated navigation states. The alignment layer propagates dynamic feasibility errors to the VLA parameters through the following physical constraint loss, enabling the model to progressively learn a velocity-curvature-adhesion consistent candidate action distribution. After fine-tuning, the same alignment layer continues to be used for online dynamic feasibility evaluation, generating dynamic masks or risk penalties based on real-time vehicle states.

[0059] The dynamic feasibility loss is used to penalize candidate trajectories for violating the velocity-curvature-adhesion coupling boundary in the prediction time domain. Inertial consistency loss is used to ensure that the posterior dynamic prediction response corresponding to the road adhesion coefficient remains consistent with the measured response of the integrated navigation system. in, For inertial consistency error, The measured vehicle acceleration output by the integrated navigation system. For vehicle dynamic response prediction function, This is the inertial observation noise covariance matrix or weight matrix.

[0060] Motion correction bias loss is used to limit the deviation between the original longitudinal motion sequence and the feasible correction result: in, This refers to the longitudinal motion sequence after dynamic feasible set projection, risk penalty correction, or safety reconstruction. The original longitudinal motion sequence generated for the VLA large model. The weight matrix for correcting deviations in the action.

[0061] Trajectory smoothing loss is used to suppress abrupt changes in longitudinal acceleration and path curvature at adjacent predicted locations: The time-domain dynamic feasible set verification process simultaneously participates in VLA parameter optimization and online candidate trajectory evaluation, enabling the model to learn to reduce the degree of dynamic boundary violation and maintain consistency between the optimization objective and online constraints.

[0062] S5. Based on the corrected action probability, perform constraint decoding and output dynamic feasible actions, safe trajectories, or control commands. The system ultimately determines the safe action probability distribution by... Greedy sampling, bundle search, or restricted sampling are performed to output a safety token or safety trajectory that satisfies the dynamic boundary constraints as the final navigation control command, which is then sent to the drive-by-wire chassis for execution. When none of the candidate actions satisfy the current safe operating domain, the system invokes a fallback strategy, such as maintaining the trajectory that was still feasible in the previous cycle, generating a controlled deceleration trajectory, or triggering an emergency braking trajectory, thereby avoiding an empty decoding set caused by dynamic masking.

[0063] Example 2 Example of low-adhesion curve calculation. In a low-adhesion curve scenario, the vehicle enters at a speed of 15 m / s with a curvature of 0.035 m. -1 For curves, the visual semantic module provides a priori information on the road surface adhesion coefficient based on the characteristics of dark ice or snow-covered surfaces. The system predicts the current vehicle acceleration response using a vehicle dynamics model based on the actual steering and braking inputs from the previous cycle, and compares this prediction with the measured acceleration output by the integrated navigation system. When the inertial response residual indicates that the actual lateral stability margin of the vehicle is less than the visual prior estimate, the correction module posteriorly corrects the road adhesion coefficient to... .

[0064] Ignoring the longitudinal acceleration's impact on the adhesion margin, the required lateral acceleration for the candidate trajectory is approximately: The upper limit of acceleration corresponding to the posterior coefficient of road adhesion is approximately: Therefore, this candidate trajectory exceeds the current predicted time-domain dynamic feasible set. Ignoring longitudinal acceleration, the theoretical safe speed upper limit obtained posteriorly from path curvature and road adhesion coefficient is: The dynamic mask processor reduces or blocks the generation probability of high-speed cornering action tokens and prioritizes candidate token prefixes that decelerate early and have a smooth steering change. If a candidate token prefix still cannot enter the dynamic feasible set within the prediction time domain, the system outputs a controlled deceleration trajectory, maintains a feasible trajectory that still satisfies the dynamic boundaries of the previous cycle, or invokes an emergency braking trajectory.

[0065] Example 3 This embodiment utilizes natural driving data and dynamics augmentation data to perform joint fine-tuning of the vision-language-action model with dynamic constraints. The dynamics augmentation data is obtained by setting low-adhesion regions, abrupt adhesion change regions, high-curvature roads, emergency obstacle avoidance targets, and different vehicle loads. Each training sample includes at least environmental observations, integrated navigation status, vehicle execution inputs, candidate actions, reference safe trajectories, road surface adhesion labels, and dynamic feasibility labels.

[0066] The candidate paths and longitudinal motion sequences generated by the vision-language-action model, along with the posterior of the road adhesion coefficient and the integrated navigation state, are input to the differentiable dynamic alignment layer. Dynamic feasibility loss penalizes candidate trajectories that exceed the velocity-curvature-adhesion coupling boundary; inertial consistency loss ensures that the predicted vehicle response corresponding to the adhesion posterior remains consistent with the time-aligned measured integrated navigation response; action correction bias loss reduces the model's dependence on online corrections and safety-degraded trajectories; and trajectory smoothing loss limits abrupt changes in longitudinal acceleration, jerk, and path curvature. The physical gradients generated by these losses are propagated to the vision-language-action model via the dynamic alignment layer, enabling the model to learn the distribution of physically executable candidate actions.

[0067] The vision-language-action model can be aligned with dynamic constraints through LoRA fine-tuning, adapter fine-tuning, cue-based optimization, or full-parameter fine-tuning. Discrete action tokens use a differentiable approximation of a dynamic mask or continuous risk penalty, while continuous trajectory output uses continuous risk penalty or differentiable projection. When the aligned model generates candidate actions online, the dynamic alignment layer updates the dynamic mask or risk penalty based on the real-time road adhesion coefficient posterior and the integrated navigation state. The calculation of each loss uses the formulas described in the invention description.

[0068] Example 4 Regarding the VLA end-to-end autonomous driving safety navigation method based on dynamic boundary constraints provided in the foregoing embodiments, this embodiment provides a VLA end-to-end autonomous driving safety navigation system based on dynamic boundary constraints, used to execute the aforementioned VLA end-to-end autonomous driving safety navigation method based on dynamic boundary constraints, such as... Figure 3 As shown, the system specifically includes: Perception and Status Acquisition Module 301: Used to synchronously acquire the historical perception feature sequence of the vehicle perception system, the current environmental image, navigation commands, and real-time physical status data output by the vehicle chassis and integrated navigation system; Dynamic envelope and adhesion estimation module 302: connected to the perception and state acquisition module, used to use a lightweight dynamic envelope regression network to fit and output the nonlinear dynamic safety operating domain of the vehicle at the current moment, and combine the visual adhesion prior and inertial response residual to obtain the road adhesion posterior. VLA candidate action generation module 303: connected to the dynamic envelope and attachment estimation module, used to receive the current environment image and structured dynamic constraint encoding, and output the original action score of the candidate action token and its corresponding candidate trajectory; Dynamic Alignment Module 304: Connected to the VLA candidate action generation module, it is used to generate physical constraint loss based on the deviation between the candidate trajectory and the predicted temporal dynamic feasible set during the training phase and backpropagate it to the vision-language-action large model. During the inference phase, it performs sequence-level dynamic verification of the candidate trajectory based on the current integrated navigation state, the posterior of the road surface adhesion coefficient and the predicted temporal dynamic feasible set, and generates dynamic masks or risk penalties.

[0069] The dynamic mask restricted decoding module 305, connected to the dynamic alignment module, is used to construct a dynamic mask matrix or risk penalty item based on the verification results, update the probability of candidate actions, and perform security degradation processing when there are no actions, outputting a compliant security trajectory or control command.

[0070] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A VLA-based end-to-end automated driving safety navigation method based on dynamic boundary constraints, characterized in that, The method includes: S1. Acquire multi-source data, including historical perception feature sequences output by the vehicle perception system, current environmental images, navigation commands, and real-time physical state data output by the vehicle chassis and integrated navigation system; S2. Input the multi-source data into a pre-trained lightweight dynamic envelope regression network, and combine the vehicle's real-time physical state, visual attachment prior, and inertial response residual correction results to end-to-end fit and output the nonlinear dynamic safety operating domain of the vehicle at the current moment. S3. The nonlinear dynamic safety operation domain is transformed into a structured dynamic constraint code, which is input into the vision-language-action large model along with the current environment image. The probability distribution of the initial navigation action sequence is generated by physical constraint reasoning, and candidate trajectories corresponding to candidate action tokens or candidate action token prefixes are generated. S4. In each autoregressive decoding step of the vision-language-action big model, the candidate trajectory is input into the dynamic alignment layer; the dynamic alignment layer takes the current integrated navigation state as the initial state, recursively calculates the vehicle response of the candidate trajectory in the prediction time domain, and constructs a dynamic mask matrix or risk penalty term based on the relationship between the candidate trajectory and the nonlinear dynamic safety operation domain to correct the generation probability of dangerous action tokens that exceed the vehicle safety dynamic boundary. S5. Based on the corrected action probability, perform constraint decoding and output dynamic feasible actions, safe trajectories, or control commands.

2. The VLA end-to-end autonomous driving safety navigation method based on dynamic boundary constraints according to claim 1, characterized in that, The nonlinear dynamic safe operating domain includes a dynamic boundary parameter vector and a predicted time-domain dynamic feasible set constructed from the dynamic boundary parameter vector. The mathematical expression is: in, It is a nonlinear mapping function. These are the network weight parameters. For a moment The acquired historical sensory feature sequence, For real-time physical state data, This represents the maximum safe curvature allowed for the vehicle under current operating conditions. This is the limiting lateral acceleration. This is the posterior estimate of the road surface adhesion coefficient. To ensure a safe speed limit in accordance with curvature and adhesion conditions, This is the matrix transpose.

3. The VLA end-to-end autonomous driving safety navigation method based on dynamic boundary constraints according to claim 2, characterized in that, The real-time physical state data includes the integrated navigation state and vehicle execution input, wherein the integrated navigation state is represented as follows: in, For longitudinal velocity, For lateral velocity, The yaw rate is angular velocity. For longitudinal acceleration, This is lateral acceleration.

4. The VLA end-to-end autonomous driving safety navigation method based on dynamic boundary constraints according to claim 2, characterized in that, The process of obtaining the posterior estimate of the road surface adhesion coefficient includes: obtaining the uncertainty of the prior and visual adhesion estimates of the road surface adhesion coefficient based on the semantics of road surface material, wetness, water accumulation, snow accumulation, or icing in the current environmental image; obtaining the predicted acceleration response using the vehicle execution input and vehicle dynamics model of the previous cycle; calculating the inertial response residual between the measured acceleration and the predicted acceleration response of the integrated navigation, and using the inertial response residual to correct the prior of the road surface adhesion coefficient, finally obtaining the online posterior estimate of the road surface adhesion coefficient.

5. The VLA end-to-end autonomous driving safety navigation method based on dynamic boundary constraints according to claim 2, characterized in that, The prediction time-domain dynamic feasible set Represented as: in, For spatial path, This is a longitudinal motion sequence generated along the spatial path. For the first Path curvature at each predicted location and These are longitudinal velocity and longitudinal acceleration, respectively. It is the acceleration due to gravity. To predict the number of discrete locations in the time domain.

6. The VLA end-to-end autonomous driving safety navigation method based on dynamic boundary constraints according to claim 1, characterized in that, The structured dynamics constraint encoding includes at least one of structured natural language prompt encoding, numerical constraint vector, or dynamic constraint embedding features; the physical constraint reasoning process includes environmental semantic perception description, underlying physical state cognition, dynamic safety boundary reasoning, and navigation action space decision-making; the structured dynamics constraint encoding is used to introduce road surface adhesion coefficient posterior, velocity-curvature constraint, longitudinal acceleration constraint, and vehicle stability margin in the candidate action token probability generation process, so that the vision-language-action large model outputs a candidate action probability distribution constrained by dynamic boundaries.

7. The VLA end-to-end autonomous driving safety navigation method based on dynamic boundary constraints according to claim 1, characterized in that, The candidate action token prefix Candidate trajectories are mapped using a preset action dictionary, trajectory decoder, or path-velocity generator. The candidate trajectory is verified point-by-point in the prediction time domain with the current integrated navigation state as the initial state; for the action decoding vocabulary The first in Candidate Action Token Define the dynamic mask function as follows: When the degree of dynamic boundary violation of the candidate trajectory is less than a preset threshold, the dynamic mask function is replaced with a finite negative risk penalty term determined according to the degree of dynamic boundary violation; when all candidate action tokens are infeasible, at least one of the following safety degradation strategies is triggered: controlled deceleration, maintaining the feasible trajectory of the previous cycle, or emergency braking trajectory.

8. The VLA end-to-end autonomous driving safety navigation method based on dynamic boundary constraints according to claim 7, characterized in that, The process of outputting dynamic action, safety trajectory, or control commands includes: Let the unnormalized log probability of the original action in the last layer of the vision-language-action large model be... After applying the dynamic masking function or risk penalty term, the probability distribution of secure actions in restricted decoding is expressed as follows: in, When the first candidate action token is a given prefix The probability of a safe action for each candidate action token. For the size of the action decoding vocabulary, , The first The and the first The original logarithmic probability of each candidate action token. , These are the dynamic masking functions or risk penalty items for the corresponding candidate action tokens; and the outputs of security token sequences, security trajectories, or control commands are achieved through greedy sampling, bundle search, or restricted sampling.

9. A VLA end-to-end automated driving safety navigation method based on dynamic boundary constraints according to claim 1, characterized in that, The vision-language-action large model is jointly trained with dynamic constraints through LoRA fine-tuning, adapter fine-tuning, cue optimization, or full parameter fine-tuning; the candidate trajectories generated by the vision-language-action model are all passed through a differentiable dynamic alignment layer. The differentiable dynamic alignment layer generates a physical gradient based on at least one of dynamic feasibility loss, inertial consistency loss, action correction deviation loss, and trajectory smoothing loss, and backpropagates the physical gradient to the visual semantic feature extraction, attachment estimation, candidate path generation, longitudinal motion generation, or action token decoding modules. After fine-tuning, the dynamic alignment layer continues to act on the online action generation process of the vision-language-action large model, and generates dynamic masks or risk penalties by combining the current integrated navigation state, the posterior of the road surface adhesion coefficient, and the predicted temporal dynamic feasible set.

10. A VLA end-to-end automated driving safety navigation system based on dynamic boundary constraints, characterized in that, The system is used to execute the VLA end-to-end autonomous driving safety navigation method based on dynamic boundary constraints as described in any one of claims 1-9, and the system includes: The perception and status acquisition module is used to synchronously acquire the historical perception feature sequence of the vehicle perception system, the current environmental image, navigation commands, and the real-time physical status data output by the vehicle chassis and integrated navigation system. The dynamic envelope and adhesion estimation module is used to output the nonlinear dynamic safety operating domain using a lightweight dynamic envelope regression network, and to obtain the posterior of the road adhesion coefficient by combining the inertial response residual. The VLA candidate action generation module is used to receive the current environment image and structured dynamics constraint encoding, and output the original action score of the candidate action token and its corresponding candidate trajectory. The dynamic alignment module is used to generate physical constraint loss based on the deviation between candidate trajectories and the predicted temporal dynamic feasible set during the training phase and backpropagate it to the vision-language-action large model. During the inference phase, it performs sequence-level dynamic verification of candidate trajectories based on the current integrated navigation state, the posterior of the road surface adhesion coefficient, and the predicted temporal dynamic feasible set, and generates dynamic masks or risk penalties. The dynamic mask restricted decoding module is used to construct a dynamic mask matrix or risk penalty item based on the verification results, update the probability of candidate actions, and perform security degradation processing when there are no actions, outputting a compliant security trajectory or control instructions.

Images