Steering control method and device for unmanned vehicle

Abstract

The invention relates to the technical field of unmanned vehicle control, and discloses a steering control method and device for an unmanned vehicle, and the method comprises the following steps: aligning multi-modal data through a dynamic calibration model, optimizing a fusion weight based on a quality index game, and solving an optimal steering instruction in combination with a vehicle state. Calibration parameters are updated through closed-loop feedback, and high-precision robust control is achieved. The system comprises a multi-source sensor calibration module, a dynamic calibration operation module, a game weight distribution module, a differential game solving module and a closed-loop feedback module. The device comprises a laser radar array, a multispectral camera group, a heterogeneous computing unit, a redundant steering actuator and a synchronous control bus. According to the method, the space-time consistency and fusion robustness of sensing data are improved through a dynamic space-time calibration model and multi-modal game optimization, the path tracking precision is optimized in combination with a differential game equation, self-correction is realized by using closed-loop feedback, and the real-time performance of the system is guaranteed by depending on a heterogeneous computing architecture.

Description

Technical Field

[0001] This invention relates to the field of autonomous vehicle control technology, and in particular to a steering control method and device for autonomous vehicles. Background Technology

[0002] With the rapid development of autonomous driving technology, multi-sensor fusion for environmental perception and high-precision steering control has become a core component for achieving autonomous vehicle navigation. Current technologies, based on multimodal perception systems using lidar, cameras, and millimeter-wave radar, generally employ static calibration methods to establish spatial mapping relationships between sensors. However, during actual driving, vehicle mechanical vibrations and random road surface excitations cause continuous shifts in sensor pose, gradually invalidating the pre-calibrated extrinsic parameter matrix and leading to spatiotemporal misalignment of multi-source data. While traditional solutions attempt dynamic compensation through inertial measurement units, they do not fully consider the coupling effect between the suspension system's vibration spectrum characteristics and road surface excitation parameters, making it difficult to maintain stable calibration accuracy under complex conditions.

[0003] At the sensor data fusion level, existing methods mostly employ fixed weights or rule-based adaptive strategies, failing to establish a closed-loop optimization mechanism for sensor quality assessment and weight allocation. When a single sensor experiences performance degradation due to environmental interference (such as rain, fog, or backlight), the fusion system is susceptible to contamination by low-quality data, leading to a decrease in the confidence level of target detection and tracking. Furthermore, the steering control stage is typically designed independently of the sensor system, failing to embed multimodal fusion weights as constraints into the control model, making it difficult to coordinate the optimization of path tracking accuracy and steering smoothness.

[0004] More significantly, existing technologies lack long-term self-calibration capabilities across sensing and control domains. Parameter drift caused by factors such as mechanical component wear and temperature fluctuations accumulates over time, causing sensor calibration models and control algorithms to gradually deviate from their optimal state. Traditional periodic manual calibration methods cannot meet the demands of continuous operation around the clock. Meanwhile, the design of real-time computing architectures for highly dynamic scenarios still has shortcomings. The timing coordination of multi-source data processing, fusion decision-making, and control command generation is insufficient, easily leading to system-level response delays. Summary of the Invention

[0005] The purpose of this invention is to provide a steering control method and device for unmanned vehicles, which solves the problems of decreased steering accuracy and robustness caused by spatiotemporal inaccuracies of multiple sensors, suboptimal fusion weight allocation, and insufficient perception-control coupling in dynamic driving environments.

[0006] To achieve the above objectives, the present invention provides a steering control method for an unmanned vehicle, the method comprising the following steps: S1. Based on the installation location information of lidar, camera and millimeter-wave radar, construct the initial external parameter matrix of multi-source sensors, and establish a dynamic spatiotemporal calibration model that integrates vehicle vibration spectrum and road excitation parameters. S2. Real-time acquisition of point cloud data from the lidar, image data from the camera, and velocity measurement data from the millimeter-wave radar; spatiotemporal consistency alignment of the multimodal data is performed using the dynamic spatiotemporal calibration model. S3. Based on the spatiotemporally aligned multimodal data, calculate the environmental perception quality index of each sensor, and dynamically allocate the fusion weights of lidar, camera and millimeter-wave radar through a multimodal game optimization algorithm. S4. Using the fusion weight as a constraint, construct a perception-control coupled differential game equation by combining the vehicle's lateral deviation, heading angle deviation and vehicle speed, solve for the optimal steering angle control command and send it to the steering actuator. S5. Based on the deviation between the actual steering angle output by the steering actuator and the optimal steering angle command, dynamically update the vibration spectrum parameters in the dynamic spatiotemporal calibration model to form a closed-loop feedback control.

[0007] Preferably, in step S1, the method for constructing the initial extrinsic parameter matrix is ​​as follows: Based on the coordinate system of each sensor's installation location, calculate its rigid body transformation matrix in the vehicle coordinate system. The rotation and translation parameters of the camera and lidar were determined using a checkerboard calibration method. The parameters of the dynamic spatiotemporal calibration model include: vibration spectrum parameter ω = [ω x ,ω y ,ω z ] T and v = [v x ,v y ,v z ] T ; Road surface excitation parameters κ and μ.

[0008] Preferably, in step S2, the calculation formula for the dynamic spatiotemporal calibration model is: in, The frequency domain vibration response matrix has a dimension of 4×4; B=κμI4 is the road excitation coupling matrix; R(θ) is the rotation matrix corresponding to the current steering angle θ; exp(·) is the Lie group exponential mapping.

[0009] Preferably, the exp(·) operation is a Lie group exponent mapping: Used to convert continuously time-varying vibration disturbances into rigid body transformation matrices.

[0010] Preferably, in step S3, the environmental perception quality indicators include: The environmental perception quality indicators include: Effective point density of lidar Where, N valid Indicates the number of valid reflection points within the obstacle's bounding box; A FOV This represents the effective detection area corresponding to the sensor's field of view. Camera gradient entropy Among them, g i p(g) represents the image gradient magnitude. i ) represents the probability of the gradient magnitude occurring within the histogram interval i; N bin Number of bins in the histogram; millimeter-wave radar signal-to-noise ratio Among them, P signal P represents the energy integral value of the target reflection peak in the Doppler spectrum. noise The average energy of the background noise; The multimodal game optimization algorithm is solved using Nash equilibrium: Among them, S i =α i ·IoU i +β i Precision i .

[0011] Preferably, in step S4: the state equation of the differential game equation is: Where, x = [e y ,e ψ ,v] T δ is the state vector; i ∈{0,1} is a sensor anomaly indicator; The design cost function is as follows: J=∫(x T Qx+u T Ru)dt; Where Q is the state weight matrix; R is the control input weight coefficient; Through the Hamiltonian function H = λ T f(x,u)+x T Qx+u T Ru solves for the optimal control quantity u * ; Where λ = [λ1, λ2, λ3] TIt is a costate vector.

[0012] Preferably, the solution process of the differential game equation includes: Constructing costate equations Update control variables using an iterative method Where η is the convergence step size.

[0013] Preferably, in step S5: the dynamic update method is gradient descent. Where η is the learning rate, and the update cycle is synchronized with the vehicle's vibration frequency.

[0014] The present invention also provides a steering control system for an unmanned vehicle, the system comprising: Multi-source sensor calibration module: configured to construct the initial external parameter matrix of each sensor in the vehicle coordinate system through three-dimensional coordinate system transformation based on the installation position of lidar, camera and millimeter-wave radar, and store the calibration data of vibration spectrum parameters and road excitation parameters; Dynamic calibration calculation module: connected to the multi-source sensor calibration module, configured to receive vehicle vibration signals and steering angle data in real time, and perform spatiotemporal alignment compensation on the raw data collected by multiple sensors based on the frequency domain response model to eliminate the pose shift caused by mechanical vibration and road bumps; Game weight allocation module: connected to the dynamic calibration calculation module, configured to analyze the quality of sensor data after spatiotemporal alignment, calculate the point cloud density of lidar, the image clarity of camera and the velocity detection accuracy of millimeter-wave radar, and dynamically allocate the fusion weights of each sensor through a multimodal game optimization algorithm; Differential game solving module: connected to the game weight allocation module, configured to use the fused weight as a constraint, combine the vehicle motion state to construct the lateral control equation, and solve the optimal control quantity that satisfies path tracking accuracy and steering smoothness through iterative optimization algorithm; Closed-loop feedback module: connected to the differential game solving module and the dynamic calibration calculation module, configured to collect the actual output angle of the steering actuator, compare it with the deviation of the theoretical control command, and dynamically adjust the vibration compensation parameters in the frequency domain response model.

[0015] The present invention also provides a steering control device for an unmanned vehicle, the device comprising: LiDAR array: installed at the four corners of the vehicle roof, with a single-point detection range of ≥200m and an angular resolution of ≤0.1°, configured to generate point cloud data of the vehicle's surrounding environment; Multispectral camera group: located at the center of the front and rear bumpers of the vehicle, including visible light and near-infrared sensing units, with a single frame image resolution ≥1920×1080 and a frame rate ≥30fps; Heterogeneous computing unit: A parallel processing architecture integrating FPGA and GPU, wherein: The FPGA chip is configured to perform dynamic spatiotemporal calibration matrix operations, including Lie group exponential mapping and vibration spectrum parameter integration; The GPU accelerator card is configured for parallel computing optimized for multimodal game theory, supporting Nash equilibrium solutions with 104 weight combinations per second; redundant steering actuator: the steering mechanism is driven by a dual-winding brushless motor, with a maximum output torque of ≥50Nm per motor, a steer-by-wire angle resolution of ≤0.01°, and a safety interlock mechanism in case of motor winding failure. Synchronous control bus: Based on time-sensitive network protocol, it realizes microsecond-level time synchronization of sensor data acquisition, calculation command transmission and actuator response.

[0016] In summary, the present invention has at least one of the following beneficial technical effects: 1. This invention constructs a dynamic spatiotemporal calibration model by fusing vehicle vibration spectrum and road excitation parameters, effectively compensating for sensor pose shifts caused by mechanical vibration and bumps, solving the problem of data inaccuracy in dynamic environments under traditional static calibration methods, ensuring the spatiotemporal synchronization of multi-source heterogeneous data from lidar, cameras and millimeter-wave radar in complex driving scenarios, and providing high-precision input for subsequent fusion and control.

[0017] 2. This invention dynamically allocates sensor fusion weights based on a multimodal game optimization algorithm. It autonomously adjusts the contribution of each sensor according to real-time perception quality indicators (such as point cloud density, image clarity, and signal-to-noise ratio). Under extreme conditions such as rain, fog, strong light, or partial sensor failure, it adaptively switches the dominant perception mode, significantly improving the system's fault tolerance and decision reliability in changing environments.

[0018] 3. This invention employs a perception-control coupled differential game equation to jointly model multiple state variables such as lateral deviation, heading angle error, and vehicle speed with sensor weight constraints, thereby achieving multi-objective collaborative optimization of steering control commands. This overcomes the limitations of the disconnect between perception and execution in traditional control methods, and balances high-precision path tracking with low-impact steering smoothness.

[0019] 4. This invention uses closed-loop feedback between the actual output of the steering actuator and the theoretical command to dynamically correct the vibration spectrum parameters and the road excitation model, overcoming the model mismatch problem caused by mechanical aging, temperature drift or changes in road characteristics in traditional open-loop calibration methods, and ensuring the dynamic calibration accuracy and control consistency of the system throughout its entire life cycle.

[0020] 5. This invention relies on a heterogeneous computing architecture and a time-sensitive network protocol to separate the computational load of dynamic calibration, game optimization and control solution, and realize end-to-end microsecond-level synchronization of multi-source data acquisition, spatiotemporal alignment, fusion decision and control command generation. This solves the timing jitter problem of traditional centralized computing architecture and meets the high real-time requirements of autonomous driving steering control. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of the method flow of the present invention; Figure 2 This is a schematic diagram of the system architecture of the present invention; Figure 3 This is a schematic diagram of the device structure of the present invention. Detailed Implementation

[0022] The following is in conjunction with the appendix Figure 1 - Appendix Figure 3 The present invention will be further described in detail below.

[0023] This invention provides a steering control method for an autonomous vehicle, such as... Figure 1 As shown, the method includes the following steps: S1. Based on the installation location information of lidar, camera and millimeter-wave radar, construct the initial external parameter matrix of multi-source sensors, and establish a dynamic spatiotemporal calibration model that integrates vehicle vibration spectrum and road excitation parameters. In this embodiment, the construction of the initial extrinsic parameter matrix and the establishment of the dynamic spatiotemporal calibration model based on multi-source sensors are specifically implemented through the following technical solutions: Initial extrinsic parameter matrix construction: Based on the physical installation positions of the LiDAR, camera, and millimeter-wave radar, a rigid body transformation model from the coordinate system of each sensor to the vehicle coordinate system is established. For each sensor, its rigid body transformation matrix in the vehicle coordinate system is calculated using the three-dimensional spatial coordinate offset of its mounting bracket. Among them, R sensor ∈SO(3) represents the rotation matrix of the sensor coordinate system relative to the vehicle coordinate system. Let be the translation vector. Preferably, for the relative pose calibration of the LiDAR and the camera, a checkerboard target method is used for joint optimization: checkerboard corner features are extracted from the LiDAR point cloud on the target plane and matched with corresponding corner points in the camera image, by minimizing the reprojection error function. Solving for the optimal rotation matrix With translation vector Ensure accurate spatial alignment of multimodal data.

[0024] Dynamic spatiotemporal calibration model construction: A six-degree-of-freedom spatiotemporal compensation model is established by integrating vehicle vibration spectrum characteristics and road excitation parameters. Specifically, this includes the following core parameter definitions: Frequency domain vibration response tensor: Based on the dynamic characteristics of the vehicle suspension system, a frequency domain response matrix characterizing the propagation law of mechanical vibration is constructed. Where, ω=[ω x ,ω y ,ω z ] T The angular velocity vector (unit: rad / s) acquired in real time by the IMU; v = [v x ,v y ,v z ] T This represents the decomposed components of the vehicle speed in the vehicle coordinate system (unit: m / s). This tensor describes the coordinate transformation disturbance caused by vibration during vehicle motion using Lie algebraic form.

[0025] Road surface excitation coupling matrix: Combining road surface roughness and tire-ground adhesion characteristics, the coupling matrix is ​​defined as: B=κ·μ·I4; Wherein, κ∈[0,1] is the road surface roughness factor, which is calculated from the variance of the ground reflection intensity distribution in the lidar point cloud data; μ is the road surface adhesion coefficient, which is estimated based on the joint estimation of tire slip ratio and longitudinal acceleration measured by millimeter-wave radar; I4 is the fourth-order identity matrix. This matrix is ​​used to quantify the coupling effect of road surface bumps on sensor pose.

[0026] Model operation principle: The continuous time-varying vibration disturbance is transformed into a discretized rigid body transformation matrix through Lie group exponential mapping. For any time t, the update formula for the spatiotemporal calibration matrix is: Where R(θ) is the steering mechanism rotation matrix corresponding to the current steering angle θ: Let be the vehicle body acceleration vector; the exp(·) operator achieves fast calculation through Chebyshev polynomial approximation. Preferably, the integration operation uses the fourth-order Runge-Kutta method to ensure numerical stability.

[0027] S2. Real-time acquisition of point cloud data from the lidar, image data from the camera, and velocity measurement data from the millimeter-wave radar; spatiotemporal consistency alignment of the multimodal data is performed using the dynamic spatiotemporal calibration model. In this embodiment, the implementation process of multimodal data spatiotemporal consistency alignment specifically includes the following technical solutions: Real-time acquisition of multi-source data: Point cloud data of the vehicle's surrounding environment is acquired via LiDAR, and the data format is a set of three-dimensional coordinates. Capture RGB image sequences using a camera It also extracts the timestamp-synchronized frame data; and obtains the radial velocity information v of the target object through millimeter-wave radar. radar (t) and distance-azimuth matrix Preferably, hardware trigger signals are used to achieve time synchronization of multi-sensor data, and the synchronization error is controlled at the microsecond level.

[0028] Application of the dynamic spatiotemporal calibration model: Based on the frequency domain vibration response tensor established in step S1 The dynamic calibration matrix is ​​calculated using the road surface excitation coupling matrix B and the real-time acquired vehicle motion parameters. The specific calculation formula is as follows: in, This is the frequency domain vibration response matrix, whose elements consist of the angular velocity component ω. x ,ω y ,ω z and linear velocity component v x ,v y ,v z Composition; B=κμI4 is the road excitation coupling matrix, κ is the road roughness factor calculated in real time, and μ is the adhesion coefficient estimated based on tire slip ratio; exp(·) operation is the Lie group exponential mapping; T is the vehicle body acceleration vector; init This is the initial extrinsic parameter matrix calibrated in step S1.

[0029] Lie group exponential mapping implementation: The exp(·) operation is a Lie group exponential mapping, whose mathematical definition is: Where A is a Lie algebra element, representing the continuously time-varying vibration disturbance. This operation converts the cumulative small pose changes caused by vehicle vibration into a discretized rigid body transformation matrix. Preferably, truncated Taylor expansion is used for real-time calculation: When ||A||<10 -2 At this time, the calculation error can be ignored.

[0030] Multimodal data transformation processing: The raw data from each sensor is transformed to a unified vehicle global coordinate system. For LiDAR point cloud data, the transformation is performed as follows: For camera image data, perspective transformation is used to eliminate image distortion caused by vehicle vibration: I rectified (t) = perspectiveTransform(I cam (t),H calib ); in, The homography matrix is ​​given by T. calib The rotation component of (t) is extracted. The millimeter-wave radar data is spatiotemporally aligned with the lidar point cloud through coordinate interpolation.

[0031] Real-time performance guarantee mechanism: the integral operation Discretized recursive calculation is used: Where Δt is the control period and k is the discrete-time index. Preferably, pre-integration techniques are used to reduce computational latency and ensure data alignment matches the sensor acquisition rate.

[0032] S3. Based on the spatiotemporally aligned multimodal data, calculate the environmental perception quality index of each sensor, and dynamically allocate the fusion weights of lidar, camera and millimeter-wave radar through a multimodal game optimization algorithm. In this embodiment, the implementation process of dynamic allocation of multimodal data fusion weights specifically includes the following technical solutions: Environmental perception quality index calculation: Based on spatiotemporally aligned multimodal data, the perception reliability of LiDAR, camera, and millimeter-wave radar is quantified separately. The following evaluation indicators are defined for the characteristics of each sensor: Effective point density of lidar: measures the distribution density of effective reflection points within the target detection area, and is calculated using the following formula: Where, N valid The number of valid reflection points within the obstacle bounding box is extracted using point cloud clustering and target tracking algorithms; A FOV The effective detection area corresponding to the sensor's field of view is calculated in real time based on the lidar's installation height and elevation angle.

[0033] Camera gradient entropy: Characterizes the richness of image features and the noise level; the calculation method is as follows: Among them, g i The gradient magnitude of the image is obtained by convolving the grayscale image using the Sobel operator; p(g i ) represents the probability of the gradient magnitude occurring within the histogram interval i; N bin The number of bins for the histogram. Preferably, adaptive threshold segmentation is used to exclude interference from regions of abrupt changes in illumination.

[0034] Millimeter-wave radar signal-to-noise ratio: Used to evaluate the anti-interference capability of radar detection signals; the calculation formula is as follows: Among them, P signal P represents the energy integral value of the target reflection peak in the Doppler spectrum. noise The average energy of the background noise is extracted using a sliding window Fourier transform.

[0035] Multimodal game optimization modeling: Constructing a non-cooperative game model to dynamically allocate sensor weights, ensuring the optimality of the fusion result under conflict scenarios. Game participants are defined as each sensor modality, and their policy space is represented by weight coefficients w. i ∈[0,1], the utility function is: U i (w i ) = w i ·S i -λ∑ j≠i w j ·D ij ; Among them, S i =α i ·IoU i +β i Precision i The overall perception score for sensor i, IoU i Precision indicates the degree of overlap between the detected region and the real target. i Historical detection accuracy; D ij The data conflict degree between sensors i and j is measured by Mahalanobis distance; α i ,β i These are the weighting coefficients determined through offline calibration.

[0036] Nash equilibrium solution: Analyzing multi-objective optimization problems using KKT conditions. Introducing the Lagrange multiplier λ to construct the dual function, and solving for the weighted closed-form solution: in, Let λ be the Lagrangian function; λ is updated iteratively until the complementary relaxation condition is satisfied. Preferably, the parallel projection gradient method is used to accelerate convergence.

[0037] Dynamic weight adjustment mechanism: The fusion weights are updated based on the real-time calculated perceived quality indicators. When ρ lidar <ρ threshold When H is affected by rain or fog, its weight is reduced; cam >H thresholdWhen the camera is deemed overexposed, the millimeter-wave radar-dominated mode is activated. Preferably, the threshold parameter ρ... threshold H threshold Adjust dynamically based on sensor performance parameters.

[0038] S4. Using the fusion weight as a constraint, construct a perception-control coupled differential game equation by combining the vehicle's lateral deviation, heading angle deviation and vehicle speed, solve for the optimal steering angle control command and send it to the steering actuator. In this embodiment, the specific implementation process of constructing the perception-control coupled differential game equation and generating the optimal steering control command is as follows: System state modeling: Based on the vehicle kinematics model and multimodal perception data, a state vector is defined to quantify the path tracking deviation: x = [e y ,e ψ ,v] T ; Among them, e y This represents the lateral deviation (in meters) between the vehicle's center of gravity and the planned path; e ψ denoted as , where is the deviation angle between the vehicle's heading angle and the tangent direction of the desired path (in radians); v is the current vehicle speed (in meters per second). The real-time values ​​of these state variables are obtained by fusing lidar point cloud matching results, visual lane line detection, and millimeter-wave radar speed measurement data.

[0039] Differential game equation construction: Under the condition of considering the weight constraints of multi-sensor fusion, a robust state equation is established: Where f(x,u) is the nonlinear dynamic equation based on the bicycle model: Where L is the vehicle wheelbase (in meters), u is the front wheel steering angle (in radians), and a is the longitudinal acceleration (in meters per second). 2 );g i (x) is the sensor error propagation function, characterizing the impact of the sensing uncertainty of the i-th sensor on state estimation; w i The fusion weights optimized for step S3; δ i ∈{0,1} represents the sensor health indicator (δ) i =1 indicates that the sensor is working properly.

[0040] Cost function design: To balance path tracking accuracy and steering smoothness, a quadratic cost function is defined: in, This is the state weight matrix, whose diagonal elements correspond to e. y,e ψ The penalty coefficient of v; To control the input weighting coefficients and suppress the rate of change of the steering angle, preferably, the Q matrix is ​​determined through Lyapunov stability analysis, and R is dynamically adjusted according to the maximum angular rate of the steering actuator.

[0041] Hamiltonian function solution: Constructing the Hamiltonian function to solve for the optimal control quantity u * : H = λ T f(x,u)+x T Qx+u T Ru; Where λ = [λ1, λ2, λ3] T Let be the costate vector. Using the Pontryagin minimum principle, the costate equation is derived: And the optimal control conditions: Iterative optimization algorithm: The gradient projection method is used to iteratively update the control variable until the convergence condition is met. Initialization: Set the initial control variable u (0) With the convergence threshold ∈, u is usually taken. (0) The previous control command; Forward integration: Integrate the state equation in the forward direction along the time domain [0,T] to obtain the state trajectory x. (k) (t); Inverse integration: Integrate the costate equation in reverse along the time domain [T,0] to calculate the costate trajectory λ. (k) (t); Control quantity update: Where η is the convergence step size, which is determined by Armijo line search to ensure the stability of the algorithm; Termination determination: when ||u (k+1) -u (k) The iteration terminates when || < ∈, and the optimal control quantity u is output. * =u (k+1) .

[0042] Control command generation: The optimized steering angle u... * This is converted into a target pulse signal for the steering actuator. Preferably, a feedforward-feedback composite control strategy is employed. Among them, K p ,K d ,K i This is the PID control gain, used to compensate for model linearization errors and external disturbances.

[0043] S5. Based on the deviation between the actual steering angle output by the steering actuator and the optimal steering angle command, dynamically update the vibration spectrum parameters in the dynamic spatiotemporal calibration model to form a closed-loop feedback control. In this embodiment, the closed-loop feedback parameter update mechanism of the dynamic spatiotemporal calibration model is specifically implemented through the following technical solution: performing residual calculation and sensitivity modeling: during the steering control cycle, the actual output steering angle δ of the steering actuator is collected in real time. real This value is obtained by measuring an optical encoder mounted on the steering column, with a measurement resolution better than 0.01°. The actual steering angle is then compared with the optimal command δ generated by the differential game solver. des By comparison, a second-order norm evaluation function for tracking bias is constructed: Where, δ des The output value of the front wheel steering angle in step S4 is the theoretical value, expressed in radians (rad). The deviation reflects the combined effects of mechanical transmission clearance, model linearization error, and external disturbances.

[0044] Mathematical implementation of gradient descent: For the vibration spectrum parameter ω=[ω in the dynamic spatiotemporal calibration model...] x ,ω y ,ω z ] T (Unit: rad / s) Establish a gradient correlation model between parameter update and tracking error. Define the parameter update equation as: ( (Learning rate); Among them, the partial derivative terms Calculated using the chain rule of error backpropagation: Partial derivatives of the control quantity with respect to the calibration matrix: This represents the sensitivity of the actual steering angle to the spatiotemporal calibration matrix. A mapping relationship is established using the kinematic model of the steering mechanism: in, It is determined by the geometric relationship between the steering rack displacement and the translation component of the calibration matrix.

[0045] Partial derivatives of the calibration matrix with respect to the vibration parameters: Let be the derivative of the Lie group matrix with respect to the vibrational spectrum parameters. Based on the derivative properties of the Lie algebraic exponential mapping, the calculation is performed by expansion: (T init ∈SE(3) is the initial extrinsic parameter matrix; in, The Jacobian matrix of the frequency domain response tensor has the following specific form: Learning rate adaptive adjustment strategy: To avoid oscillations or divergences during parameter updates, the learning rate η is designed to be related to the oscillation energy E. vib Dynamic association rules: η=η0·exp(-γE vib ),( (The attenuation coefficient); Among them, vibration energy E vib Calculated by time-domain integration of the vehicle body acceleration signal: (T is the length of the integration window, which is usually taken as 2-3 times the principal period of the vibration); When a violent vibration (E) is detected vib >E threshold When η is reduced, it automatically decreases to suppress parameter mutations; under stable operating conditions, it restores a larger learning rate to accelerate convergence.

[0046] Determination of the update period for vibration frequency synchronization: parameter update period T update With the vehicle vibration dominant frequency f dominant Real-time synchronization, the specific calculation formula is as follows: ( (Usually n = 1 or 2); Vibration dominant frequency f dominant via vehicle acceleration signal The Fast Fourier Transform (FFT) spectral analysis yielded the following: right Perform a windowed FFT to obtain the spectrum. Extract the frequency f corresponding to the peak value of the spectrum. peak =argmax f S(f); If f peak Within the natural frequency range of the vehicle suspension system (typically 1-10Hz), f is determined. dominant =f peak Otherwise, use the historical frequency value.

[0047] Integrity verification of the closed-loop update process: Parameter update triggered: when T is reached update At that time, the gradient descent calculation process is initiated; Lie group matrix orthogonality check: Updated T calib Special Euclidean group constraints must be satisfied: If the verification fails, the Gram-Schmidt orthogonalization method is used to correct the rotation matrix components. Historical data backtracking: Saves the most recent N parameter update records (usually N=100). When a parameter mutation is detected (||Δω||>ω), threshold When this happens, it will automatically roll back to a stable version; Actuator command smoothing: To avoid control chattering caused by parameter jumps, the updated ω is low-pass filtered: ω filtered =(1-α)ω prev +αω new (α∈[0,1] are the filter coefficients).

[0048] The steering control method system for an autonomous vehicle described below can be referred to in correspondence with the steering control method for an autonomous vehicle described above.

[0049] Please see the appendix Figure 2 This invention also provides a steering control system for an autonomous vehicle, comprising: a multi-source sensor calibration module deployed during the vehicle's computing unit initialization phase, responsible for establishing a multi-sensor spatial reference. Based on the physical installation parameters of lidar, cameras, and millimeter-wave radar, the transformation relationship from each sensor to the vehicle coordinate system is constructed using the principles of three-dimensional rigid body kinematics, generating an initial extrinsic parameter matrix library. An internal non-volatile memory stores vibration spectrum characteristic parameters and road excitation coupling coefficients, including calibration data such as suspension system resonant frequency and tire stiffness coefficient, providing a reference parameter set for dynamic calibration.

[0050] Dynamic calibration computation module: Integrates signal processing unit and motion compensation algorithm, and interfaces in real time with vehicle inertial measurement unit, steering angle sensor and multi-source sensing data stream. By analyzing the vehicle vibration acceleration spectrum characteristics and combining road bump excitation model, it generates a six-degree-of-freedom spatiotemporal transformation matrix to compensate online for LiDAR point cloud distortion, camera image jitter and millimeter-wave data offset. A sliding window mechanism is used to update pose transformation parameters to ensure the spatiotemporal consistency of multimodal data in dynamic environments.

[0051] The game-theoretic weight allocation module is equipped with a multimodal quality assessment engine and a game theory optimizer, receiving spatiotemporally aligned sensor data streams. Feature extraction channels are designed separately for LiDAR point clouds, camera images, and millimeter-wave signals, quantifying and calculating quality indicators such as effective reflectivity of point clouds, image edge sharpness, and velocity detection confidence. A non-cooperative game model is constructed to analyze the competitive and cooperative relationships between sensors, and the optimal fusion weights are generated through Nash equilibrium solving, enabling adaptive allocation of sensing resources in complex environments.

[0052] Differential game theory solution module: Embedded with a real-time optimal controller, it receives fused weight constraints and vehicle motion state feedback. Based on a lateral dynamics model, it constructs a path tracking error equation and couples sensor weight influence factors to establish a robust optimization objective function. A numerical iterative algorithm is used to solve for the optimal control sequence that satisfies multiple objective conditions such as steering angle rate constraints and lateral acceleration limits, generating steering angle commands that balance tracking accuracy and ride comfort.

[0053] Closed-loop feedback module: Constructs a data feedback channel across the sensing and control domains to monitor the actual angular displacement output of the steering actuator in real time. It collects the response delay and nonlinear error of the mechanical transmission system using a high-precision angle sensor, and calculates the deviation spectrum between theoretical commands and actual outputs. Based on a gradient backpropagation mechanism, it dynamically corrects the frequency domain response parameters in the vibration compensation model, suppressing model mismatch caused by mechanical wear and temperature drift, and ensuring calibration accuracy under all operating conditions.

[0054] System coordination mechanism: Each module achieves microsecond-level data synchronization via a time-sensitive bus. The dynamic calibration calculation module updates spatiotemporal transformation parameters at a 5ms cycle, the game weight allocation module refreshes the fusion weights at a 20ms cycle, and the differential game solving module outputs control commands at a 10ms cycle. The anomaly handling unit monitors the module's operating status in real time. When sensor failure or communication timeout is detected, it automatically switches to degraded control mode and triggers a safety alarm.

[0055] The steering control method and device for an autonomous vehicle described below can be referred to in correspondence with the steering control method for an autonomous vehicle described above.

[0056] Please see the appendix Figure 3 This invention also provides a steering control device for an autonomous vehicle, comprising: a lidar array: installed in a distributed layout at the four corners of the vehicle roof to form 360° coverage without blind spots. Each lidar is equipped with a time-of-flight (ToF) ranging unit and a two-dimensional scanning galvanometer, achieving high-precision point cloud acquisition with a single-point detection distance of not less than 200 meters and a horizontal angular resolution better than 0.1°. A dense three-dimensional semantic map of the vehicle's surrounding environment is generated through a multi-liquidity lidar point cloud fusion algorithm, supporting dynamic obstacle trajectory prediction and static curb feature extraction.

[0057] Multispectral camera array: Symmetrically positioned at the center of the front and rear bumpers of the vehicle, this visible-near-infrared dual-band imaging system employs a common optical path design. The visible light unit is equipped with a global shutter CMOS sensor, achieving a single-frame resolution of no less than 1920×1080 pixels, supporting HDR imaging to handle scenarios with changing light conditions such as tunnel entrances and exits. The near-infrared unit integrates a 940nm band filter, actively supplementing light to enhance image contrast in low-light or rain / fog environments. The dual-spectral data, after spatiotemporal alignment, is input into a multimodal feature extraction network to achieve all-weather recognition of lane lines and traffic signs.

[0058] Heterogeneous Computing Unit: A hybrid computing architecture integrating a programmable gate array (FPGA) and a graphics processing unit (GPU). The FPGA chip incorporates parallel pipeline logic dedicated to performing exponential mapping operations on the dynamic spatiotemporal calibration matrix, integration of frequency domain vibration parameters, and real-time compensation calculations for Lie group transforms, ensuring a spatiotemporal calibration latency of less than 100 microseconds. The GPU accelerator card features a multi-threaded optimized architecture, performing parallel searches of the weight combination space for multimodal game optimization, supporting fast solutions for Nash equilibrium strategies exceeding 10,000 times per second, meeting the real-time decision-making requirements of complex scenarios.

[0059] Redundant steering actuator: Utilizing a dual-winding brushless motor to drive the steering rack mechanism, the maximum output torque of a single winding is no less than 50 Nm. An integrated high-precision magnetic encoder enables closed-loop control of the steering angle, with a resolution better than 0.01°. In the event of a short circuit or open circuit fault in either winding, a safety interlock mechanism immediately switches to power from the backup winding and maintains steering torque output based on Hall sensor feedback. The actuator controller integrates overheat protection and stall detection functions to ensure safe reversal under extreme conditions.

[0060] Synchronous Control Bus: A full-vehicle data transmission link is constructed based on the Time-Sensitive Networking (TSN) protocol, and clock synchronization of various sensors, computing units, and actuators is achieved through the IEEE 802.1AS standard. The bus supports microsecond-level deterministic transmission, ensuring that the alignment error of the acquisition timestamps of LiDAR point clouds, camera images, and millimeter-wave data is less than 10 microseconds, and the end-to-end latency from control command generation to execution is less than 5 milliseconds.

[0061] Device collaboration mechanism: Raw data from the lidar and camera are transmitted to the heterogeneous computing unit via a synchronous control bus. After spatiotemporal calibration by the FPGA, the data is input into the GPU for multimodal fusion. The weight parameters generated by game optimization are sent to the differential game solving module in real time via the TSN bus to drive the redundant steering actuator to perform high-precision steering control. The closed-loop feedback module continuously monitors the actuator status, dynamically adjusts the calibration parameters, and sends them back to the FPGA to realize online model updates.

[0062] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A steering control method for an unmanned vehicle, characterized in that, The method includes the following steps: S1. Based on the installation location information of lidar, camera and millimeter-wave radar, construct the initial external parameter matrix of multi-source sensors, and establish a dynamic spatiotemporal calibration model that integrates vehicle vibration spectrum and road excitation parameters. S2. Real-time acquisition of point cloud data from the lidar, image data from the camera, and velocity measurement data from the millimeter-wave radar; spatiotemporal consistency alignment of the multimodal data is performed using the dynamic spatiotemporal calibration model. S3. Based on the spatiotemporally aligned multimodal data, calculate the environmental perception quality index of each sensor, and dynamically allocate the fusion weights of lidar, camera and millimeter-wave radar through a multimodal game optimization algorithm. S4. Using the fusion weight as a constraint, construct a perception-control coupled differential game equation by combining the vehicle's lateral deviation, heading angle deviation and vehicle speed, solve for the optimal steering angle control command and send it to the steering actuator. S5. Based on the deviation between the actual steering angle output by the steering actuator and the optimal steering angle command, dynamically update the vibration spectrum parameters in the dynamic spatiotemporal calibration model to form a closed-loop feedback control.

2. The steering control method for an unmanned vehicle according to claim 1, characterized in that, In step S1: the method for constructing the initial extrinsic parameter matrix is ​​as follows: Based on the coordinate system of each sensor's installation location, calculate its rigid body transformation matrix in the vehicle coordinate system. The rotation and translation parameters of the camera and lidar were determined using a checkerboard calibration method. The parameters of the dynamic spatiotemporal calibration model include: vibration spectrum parameter ω = [ω x ,ω y ,ω z ] T and v = [v x ,v y ,v z ] T ; Road surface excitation parameters κ and μ.

3. The steering control method for an unmanned vehicle according to claim 1, characterized in that, In step S2: the calculation formula for the dynamic spatiotemporal calibration model is: in, The frequency domain vibration response matrix has a dimension of 4×4; B=κμI4 is the road excitation coupling matrix; R(θ) is the rotation matrix corresponding to the current steering angle θ; exp(·) is the Lie group exponential mapping.

4. The steering control method for an unmanned vehicle according to claim 3, characterized in that, The exp(·) operation is a Lie group exponent mapping: Used to convert continuously time-varying vibration disturbances into rigid body transformation matrices.

5. The steering control method for an unmanned vehicle according to claim 1, characterized in that, In step S3: the environmental perception quality indicators include: The environmental perception quality indicators include: Effective point density of lidar Where, N valid Indicates the number of valid reflection points within the obstacle's bounding box; A FOV This represents the effective detection area corresponding to the sensor's field of view. Camera gradient entropy Among them, g i p(g) represents the image gradient magnitude. i ) represents the probability of the gradient magnitude occurring within the histogram interval i; N bin Number of bins in the histogram; millimeter-wave radar signal-to-noise ratio Among them, P signal P represents the energy integral value of the target reflection peak in the Doppler spectrum. noise The average energy of the background noise; The multimodal game optimization algorithm is solved using Nash equilibrium: Among them, S i =a i ·IoU i +b i ·Precision i 。 6. The steering control method for an unmanned vehicle according to claim 1, characterized in that, In step S4: the state equation of the differential game equation is: Where, x = [e y ,e ψ ,v] T δ is the state vector; i ∈{0,1} is a sensor anomaly indicator; The design cost function is as follows: J=∫(x T Qx+u T Ru)dt; Where Q is the state weight matrix; R is the control input weight coefficient; Through the Hamiltonian function H = λ T f(x,u)+x T Qx+u T Ru solves for the optimal control quantity u * ; Where λ = [λ1, λ2, λ3] T It is a costate vector.

7. The steering control method for an unmanned vehicle according to claim 6, characterized in that, The solution process for the differential game equation includes: Constructing costate equations Update control quantity using iterative method Where η is the convergence step size.

8. The steering control method for an unmanned vehicle according to claim 1, characterized in that, In step S5: the dynamic update method is gradient descent. Where η is the learning rate, and the update cycle is synchronized with the vehicle's vibration frequency.

9. A steering control system for an unmanned vehicle, applied to the method described in any one of claims 1-8, characterized in that, The system includes: Multi-source sensor calibration module: configured to construct the initial external parameter matrix of each sensor in the vehicle coordinate system through three-dimensional coordinate system transformation based on the installation position of lidar, camera and millimeter-wave radar, and store the calibration data of vibration spectrum parameters and road excitation parameters; Dynamic calibration calculation module: connected to the multi-source sensor calibration module, configured to receive vehicle vibration signals and steering angle data in real time, and perform spatiotemporal alignment compensation on the raw data collected by multiple sensors based on the frequency domain response model to eliminate the pose shift caused by mechanical vibration and road bumps; Game weight allocation module: connected to the dynamic calibration calculation module, configured to analyze the quality of sensor data after spatiotemporal alignment, calculate the point cloud density of lidar, the image clarity of camera and the velocity detection accuracy of millimeter-wave radar, and dynamically allocate the fusion weights of each sensor through a multimodal game optimization algorithm; Differential game solving module: connected to the game weight allocation module, configured to use the fused weight as a constraint, combine the vehicle motion state to construct the lateral control equation, and solve the optimal control quantity that satisfies path tracking accuracy and steering smoothness through iterative optimization algorithm; Closed-loop feedback module: connected to the differential game solving module and the dynamic calibration calculation module, configured to collect the actual output angle of the steering actuator, compare it with the deviation of the theoretical control command, and dynamically adjust the vibration compensation parameters in the frequency domain response model.

10. A steering control device for an unmanned vehicle, applied to the method described in any one of claims 1-8, characterized in that, The device includes: LiDAR array: installed at the four corners of the vehicle roof, with a single-point detection range of ≥200m and an angular resolution of ≤0.1°, configured to generate point cloud data of the vehicle's surrounding environment; Multispectral camera group: located at the center of the front and rear bumpers of the vehicle, including visible light and near-infrared sensing units, with a single frame image resolution ≥1920×1080 and a frame rate ≥30fps; Heterogeneous computing unit: A parallel processing architecture integrating FPGA and GPU, wherein: The FPGA chip is configured to perform dynamic spatiotemporal calibration matrix operations, including Lie group exponential mapping and vibration spectrum parameter integration; The GPU accelerator card is configured for parallel computing optimized for multimodal game theory, supporting 10 kilobytes per second. 4 Nash equilibrium solution for sub-weighted combinations; Redundant steering actuator: The steering mechanism is driven by a dual-winding brushless motor with a maximum output torque of ≥50Nm per motor and a steer-by-wire angle resolution of ≤0.01°. It has a safety interlock mechanism in case of motor winding failure. Synchronous control bus: Based on time-sensitive network protocol, it realizes microsecond-level time synchronization of sensor data acquisition, calculation command transmission and actuator response.

Images