Active suspension preview control method based on road perception and hybrid synchronization strategy

Abstract

The invention relates to the technical field of vehicle intelligent control, and provides an active suspension preview control method based on pavement perception and a hybrid synchronization strategy, and the method comprises the steps: obtaining image data and laser radar point cloud data of a front road; processing the image data and the point cloud data by adopting a hierarchical road sensing module, and extracting structured road state information; splicing the road state information with the current physical state information of the vehicle suspension system to form an augmented observation state; and the reinforcement learning control module based on the hybrid synchronization strategy outputs the optimal active control force according to the augmented observation state so as to regulate and control the suspension actuator in real time. According to the method, the preview information of the front road can be efficiently and stably utilized, then the active suspension control strategy is constructed, and the riding and driving comfort of the vehicle is remarkably improved under various complex road conditions.

Description

Technical Field

[0001] This invention relates to the field of vehicle intelligent control technology, and more specifically, to an active suspension anti-aiming control method based on road perception and a hybrid synchronization strategy. Background Technology

[0002] The vehicle suspension system is a key subsystem of the chassis. Its main function is to dampen body vibrations caused by road surface unevenness, playing a decisive role in the vehicle's handling stability, ride comfort, and overall ride quality. Traditional suspension control strategies (such as PID and LQR) and robust control methods (such as H∞ control and sliding mode control) primarily rely on feedback from the current suspension state for passive or semi-active adjustments. Because these methods cannot utilize information from the road ahead, they inherently exhibit lag when responding to sudden or periodic road surface excitations, limiting further improvements in ride comfort.

[0003] To overcome the aforementioned limitations, anti-sight control technology has been introduced into the field of active suspension. This technology uses onboard sensors (such as cameras and lidar) to acquire information about the road contour ahead in advance and makes proactive adjustments accordingly. Existing anti-sight control methods typically employ model predictive control (MPC), but its performance is highly dependent on an accurate vehicle dynamics model and has high computational complexity, making it difficult to adapt to real-world operating conditions with time-varying parameters and strong nonlinearities.

[0004] In recent years, deep reinforcement learning (DRL) has been widely applied to active suspension control due to its powerful model independence and ability to make autonomous decisions in complex environments. However, most existing DRL methods only make reactive decisions based on the current suspension state, failing to effectively utilize pre-aimed road information. Even studies attempting to directly stitch road elevation data into the state input suffer from limited performance gains due to a lack of effective representation and fusion of multimodal road information (such as image semantics and point cloud geometry). Furthermore, DRL algorithms themselves often face the challenge of balancing sample efficiency and training stability during training: on-policy algorithms (such as PPO) are stable but have low sample efficiency; off-policy algorithms (such as TD3 and SAC) have high sample efficiency but are susceptible to overestimation due to function approximation errors, leading to training instability.

[0005] Therefore, there is an urgent need for a reinforcement learning control framework that can efficiently integrate multi-source road perception information and design a framework that can simultaneously take into account training stability and sample efficiency, so as to fully leverage the potential of pre-aiming information in active suspension control. Summary of the Invention

[0006] In view of this, the present invention proposes an active suspension anti-aiming control method based on road surface perception and hybrid synchronization strategy, which aims to significantly improve the driving smoothness and ride comfort of vehicles under complex road conditions.

[0007] To achieve the above objectives, this invention proposes an active suspension anti-aiming control method based on road surface perception and a hybrid synchronization strategy, characterized by the following steps: Acquire image data and lidar point cloud data of the road ahead; The image data and point cloud data are processed using a layered road perception module to extract structured road state information; The road condition information is spliced ​​with the current physical condition information of the vehicle suspension system to form an augmented observation state; The reinforcement learning control module based on the hybrid synchronization strategy outputs the optimal active control force according to the augmented observation state, so as to adjust the suspension actuator in real time. The hybrid synchronous strategy reinforcement learning control module adopts a dynamically coupled same-policy learning and different-policy learning method, and uses a synchronization mechanism to periodically align the learning objectives of the same-policy learning and different-policy learning methods.

[0008] Furthermore, the road status information includes at least a road category label and the elevation value of the road ahead; The current physical state information of the vehicle suspension system includes sprung mass acceleration, velocity, displacement, and suspension dynamic deflection.

[0009] Furthermore, the hierarchical road perception module includes a road classification unit and a road elevation estimation unit set in parallel; The road classification unit is used to process the image data using an improved lightweight convolutional neural network and output road category labels. The road elevation estimation unit is used to process the lidar point cloud data using a Kalman filter-enhanced random sampling consensus algorithm, fit the local road plane, and calculate the elevation value of the road ahead.

[0010] Furthermore, the improved lightweight convolutional neural network adopts a hybrid attention-guided one-dimensional feature extraction mechanism. For multi-scale features, it first fuses channel and spatial attention to generate a unified mask, then performs attention-guided spatial max pooling to retain highly discriminative local information, and combines a multi-head classifier to output road category labels.

[0011] Furthermore, the Kalman filter-enhanced random sample consensus algorithm specifically processes the following: A state-space model with road plane equation coefficients as state vectors is constructed. Point cloud data between time frames are fused using a Kalman filter. The plane parameter estimate of the previous time step is used as a priori, and the plane parameters obtained by fitting the current frame through the RANSAC algorithm are used as observations. Recursive state updates are performed to obtain local road plane estimates. Road elevation values ​​are then analytically calculated based on the local road plane estimates.

[0012] Furthermore, the hybrid synchronization strategy reinforcement learning control module achieves dynamic coupling learning in the following way: Set up a round buffer and a replay buffer to store the complete interaction trajectory and single-step transition samples across rounds, respectively; During the same policy learning phase, based on the complete trajectory in the round buffer, the weighted multi-round expected reward is calculated, and the policy network and value network are updated based on the policy gradient method. During the heterogeneous policy learning phase, historical data is periodically sampled from the replay buffer to update the Q-function ensemble and policy network. The Q-network ensemble is used to optimize the most conservative Q-value, and a Q-distribution network is introduced to impose distribution constraints to mitigate overestimation. Through a synchronization mechanism, the randomly selected Q-network estimate is aligned with the multi-round expected reward calculated in the same policy phase during each training cycle, so as to achieve synchronization of the learning objectives of the two learning paradigms.

[0013] Furthermore, the weighted multi-round expected return The calculation formula is:

[0014] in, , , These represent the k-step rewards calculated at time step t for the current round and the two previous historical rounds, respectively. , , These are learnable weight parameters.

[0015] Further, in step S4, the action space is adaptively divided according to the road category label, specifically as follows: The original action output by the policy network is compressed by the tanh function and then multiplied by a scaling factor related to the current road category to obtain the final control force acting on the actuator. For rough roads, the scaling factor is larger, and the actuator outputs a greater force. For smooth roads, the scaling factor is smaller, and the actuator outputs a smaller force.

[0016] Furthermore, the hybrid synchronous strategy reinforcement learning control module employs a soft normalization method to design the reward function, specifically including: The state variables are normalized using the extreme values ​​of the passive suspension response under typical excitation as the normalization benchmark; the reward function includes an overflow penalty imposed on states that exceed the response range of the passive system, a smooth negative reward imposed on key performance indicators, and a sparse positive reward imposed on acceleration and suspension deflection below a certain threshold.

[0017] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention utilizes a hierarchical perception module, enabling the controller to simultaneously acquire semantic category and geometric elevation information of the road, thus achieving pre-aiming control. Experiments show that, compared to common passive suspensions, this invention can reduce the root mean square value of the vehicle's vertical acceleration by 51.7% to 75.5%. This outperforms other advanced DRL algorithms (CQL, TD3) and MPC methods by up to 12.7%.

[0018] The improved MobileNetV3 and Kalman filter-enhanced RANSAC algorithms of this invention guarantee high-precision classification of road images and smooth, robust estimation of point cloud elevation under resource-constrained conditions, respectively.

[0019] The hybrid synchronization strategy proposed in this invention combines the advantages of on-policy and off-policy methods, ensuring the stability of policy updates while making full use of historical data, significantly improving sample utilization efficiency, and enabling the control policy to learn the optimal control policy faster and more reliably.

[0020] This invention uses road category-guided adaptive partitioning of the action space to enable the control strategy to focus on more relevant action ranges under different road conditions, thereby accelerating strategy convergence. Attached Figure Description

[0021] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. In the drawings: Figure 1 This is a schematic diagram of the overall framework of the active suspension anti-aiming control method based on road perception and hybrid synchronization strategy of the present invention. Figure 2 This invention uses the improved MobileNetV3 lightweight neural network architecture. Detailed Implementation

[0022] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the disclosure to those skilled in the art. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0023] This embodiment proposes an active suspension anti-aiming control method based on a layered road perception and hybrid synchronization strategy, such as... Figure 1 As shown, the core of this method lies in constructing a perception-decision integrated control framework. Figure 1 The system consists of a hierarchical road perception module and a hybrid synchronization strategy reinforcement learning control module. The former fuses camera images and LiDAR point clouds to output a structured road state (including semantic categories and local elevations), and then concatenates this road state with the suspension physical state to form an augmented observation input. The latter receives this state input and generates active control forces through a hybrid synchronization strategy. The framework includes two types of experience buffers—a round buffer to store complete trajectories to support on-policy learning, and a replay buffer to store single-step transition samples across rounds to support off-policy learning.

[0024] A. Layered road perception module: An improved lightweight convolutional neural network (MobileNetV3) is used to process images of the road ahead, outputting discrete road category labels (such as "flat" and "pothole"). This network abandons traditional global average pooling (GAP) and instead employs an efficient hybrid attention-guided one-dimensional feature extraction mechanism, combined with a multi-head classifier to preserve more discriminative spatial detail. Its structure is as follows: Figure 2 As shown, in Figure 2 In this process, the input road image is processed by the backbone network to extract multi-scale features. First, channel and spatial attention are fused to generate a unified mask. Then, attention-guided spatial max pooling is performed to retain highly discriminative local information. Finally, the feature vectors are aggregated by a multi-head classifier to output robust road category labels, providing high-level semantic pre-aiming information for the control strategy.

[0025] For lidar point cloud data, a Kalman filter-enhanced Random Sample Consensus (RANSAC) algorithm is applied to the tire contact area to fit a robust and smooth local road plane and calculate the accurate elevation value of the road ahead. This method effectively suppresses estimation jitter caused by single-frame point cloud noise and sparsity by fusing temporal inter-frame information through Kalman filtering.

[0026] The hierarchical road perception module contains two parallel processing units: The road classification unit employs an improved lightweight convolutional neural network (such as MobileNetV3) to process the input road images. This network abandons the traditional Global Average Pooling (GAP) operation, instead using an efficient one-dimensional feature extraction mechanism. This mechanism first generates a unified attention mask through channel attention and spatial attention, then performs attention-guided spatial max pooling to preserve the most discriminative local features, ultimately generating a compact and highly discriminative one-dimensional feature vector. Building upon this, a multi-head classification architecture is introduced, learning different semantic patterns through multiple independent classification heads. The outputs of each head are averaged to obtain the final road category label (e.g., smooth, pothole). This road category information provides a high-level semantic reference for subsequent policy learning.

[0027] Road Elevation Estimation Unit: A Kalman filter-enhanced RANSAC (Random Sample Consensus) algorithm is used to process LiDAR point cloud data. Traditional RANSAC algorithms process each frame of point cloud independently, while this invention constructs a state-space model, using the coefficients (a, b, c, d) of the road plane equation as the state vector. The Kalman filter utilizes temporal continuity, using the previous time-series plane parameter estimates as priors and the fitting result of the current frame's RANSAC as the observation, recursively updating the state. This method effectively suppresses the influence of noise and outliers, generating a smooth and robust local road plane estimate. Based on this fitted plane, the accurate road elevation value within the tire contact area can be directly and analytically calculated.

[0028] The outputs of the two units together constitute a structured road state representation.

[0029] B. Hybrid Synchronous Strategy Reinforcement Learning Control Module: The core of this invention lies in the proposed hybrid synchronous policy reinforcement learning algorithm. This algorithm balances training stability and sample efficiency by dynamically coupling on-policy and off-policy learning paradigms.

[0030] State and Action Construction: The road category labels and road elevation values ​​output by the hierarchical perception module are concatenated with the physical states of the suspension system (such as sprung mass acceleration, velocity, displacement, and suspension deflection) to form the augmented state input for the reinforcement learning strategy. The action output is the control force of the actuator, the amplitude of which is adaptively scaled according to the current road category, realizing the explicit partitioning of the action space.

[0031] Hybrid Synchronous Learning Mechanism: A novel hybrid synchronous strategy algorithm is designed to dynamically couple on-policy and off-policy learning paradigms.

[0032] On-policy phase: Using the complete round trajectory, the cumulative return is calculated using the Monte Carlo method. To reduce variance, a learnable weight is introduced to perform a weighted average of the returns of the current round and the previous two rounds, resulting in a more stable return estimate.

[0033] Off-policy phase: Historical transition data from the experience replay buffer is used to update the Q-value using the temporal difference (TD) method. To mitigate Q-value overestimation, a Q-value distribution network is introduced to learn the probability density of the Q-value, and distribution constraints are imposed to suppress overestimation of out-of-distribution (OOD) actions. Simultaneously, a Q-network ensemble is employed, and the most conservative Q-value is optimized to further improve the policy's robustness.

[0034] Synchronization Mechanism: At the end of each training cycle, a network is randomly selected from the Q-network ensemble, and its Q-value estimate is aligned with the weighted reward calculated in the on-policy phase. This operation achieves periodic synchronization of the two learning objectives, enabling the off-policy phase to provide stable supervision signals for the on-policy phase, while the on-policy phase provides an unbiased reward benchmark for the off-policy phase, thus balancing training stability and sample efficiency overall.

[0035] The specific implementation steps of the algorithm are as follows: Step S1: Initialization and Data Buffering Initialize policy network State-value network Q-function integration { } (containing n independent Q-networks), Q-distributed network .

[0036] Establish two experience replay buffers: a round buffer. Used to store the complete trajectory; replay buffer Used to store single-step transfer samples spanning multiple rounds. .

[0037] Step S2: On-policy Phase In each training round, the current policy Interact with the environment and store the complete turn trajectory. .

[0038] After the round ends, calculate each time step in that round. of Step Expected Return This expected return is not the return of a single round, but rather a weighted sum of the returns of the current round and the two previous rounds:

[0039] Among them, weight , , These are learnable parameters.

[0040] Using this low-variance unbiased return estimate and state value function As a baseline, the policy network is updated using the policy gradient method. and value network This stage ensures the unbiasedness and stability of strategy updates.

[0041] Step S3: Off-policy Phase Periodically (every 5 rounds) from the replay buffer Training is performed using a batch of historical data sampled from the middle.

[0042] Q-function ensemble learning: uses a standard temporal difference (TD) objective, combined with a distribution-constrained loss, to update the Q-function ensemble { This distribution constraint is achieved through an independent Q-distribution network. The network learns a Gaussian distribution of Q-values. Overestimation of Q-values ​​is mitigated by penalizing out-of-distribution actions.

[0043] Policy optimization: When updating the policy, the most conservative Q-value (i.e., the minimum value in the Q-ensemble) is used as the optimization objective. This further enhances the robustness of the strategy.

[0044] Synchronization mechanism: At the start of each dissimilar policy update, a Q-network is randomly selected from the Q-ensemble, and its Q-value estimate is compared with the expected return calculated in the same policy phase. Alignment is performed (by minimizing the mean squared error). This operation achieves dynamic coupling between the two learning paradigms: It provides a stable monitoring signal from a large amount of historical data for estimating returns at the same strategy stage.

[0045] Use the aligned Q-value to dynamically update the weights in the expected return. , , This allows the contributions of historical rounds to be adjusted adaptively.

[0046] Through the aforementioned periodic synchronization, this algorithm successfully combines the stability of same-policy learning with the high sample efficiency of different-policy learning.

[0047] C. Action space adaptation and reward function design: A soft normalization method is employed to design the reward function. First, the extreme response of the passive suspension under typical excitation is used as the normalization benchmark, and the state variables are pruned and linearly scaled. The reward function comprises several components: an overflow penalty is imposed on states exceeding the response range of the passive system; a smooth negative reward is imposed on sprung mass acceleration, suspension relative velocity, and dynamic deflection; a hard penalty is imposed on excessive acceleration; and simultaneously, sparse positive rewards are given when acceleration and suspension dynamic deflection are below a certain threshold to guide the strategy towards exploring the high-performance region.

[0048] The specific implementation steps are as follows: Action space adaptation: the original action output by the agent has undergone... Function compression to After the interval, it will be multiplied by a scaling factor that depends on the road type. For more challenging roads (such as potholes), this factor is larger, allowing the actuator to output greater force; for smoother roads, the factor is smaller. This adaptive design ensures both the effectiveness of control and improves the efficiency of the strategy in exploring different road conditions.

[0049] Reward Function: A soft normalization method is proposed to design the reward function. First, the state variables are normalized using the extreme values ​​of the passive suspension's response under typical road conditions. The reward function comprehensively considers multiple indicators: smoothing penalties are applied to key indicators such as sprung mass acceleration and suspension dynamic deflection; overflow penalties are imposed on states exceeding the passive suspension's response range; and sparse positive rewards are set to encourage low acceleration (comfort) and small deflection (safety). Furthermore, the original reward is time-smoothed to stabilize the learning process.

[0050] In summary, the implementation process of this embodiment can be summarized as follows: First, onboard sensors (such as cameras and LiDAR) acquire real-time image and point cloud data of the road ahead. Then, a hierarchical road perception module processes this multimodal data to extract structured road state information. This road state information, together with the current physical state of the vehicle's suspension system (such as sprung mass acceleration, velocity, displacement, and suspension deflection), constitutes the augmented observation state for the reinforcement learning control strategy. Finally, a reinforcement learning algorithm based on a hybrid synchronization strategy outputs the optimal active control force based on this augmented observation state to achieve real-time regulation of the suspension actuators, thereby maximizing ride comfort.

[0051] Through the above-described embodiments, the present invention can efficiently and stably train an active suspension control strategy that can make full use of the forward road preview information, significantly improving the vehicle's ride comfort under various complex road conditions.

[0052] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. An active suspension anti-aiming control method based on road surface perception and hybrid synchronization strategy, characterized in that, Includes the following steps: Acquire image data and lidar point cloud data of the road ahead; The image data and point cloud data are processed using a layered road perception module to extract structured road state information; The road condition information is spliced ​​with the current physical condition information of the vehicle suspension system to form an augmented observation state; The reinforcement learning control module based on the hybrid synchronization strategy outputs the optimal active control force according to the augmented observation state, so as to adjust the suspension actuator in real time. The hybrid synchronous strategy reinforcement learning control module adopts a dynamically coupled same-policy learning and different-policy learning method, and uses a synchronization mechanism to periodically align the learning objectives of the same-policy learning and different-policy learning methods.

2. The method according to claim 1, characterized in that, The road status information includes at least a road category label and the elevation value of the road ahead; The current physical state information of the vehicle suspension system includes sprung mass acceleration, velocity, displacement, and suspension dynamic deflection.

3. The method according to claim 1, characterized in that, The hierarchical road perception module includes a road classification unit and a road elevation estimation unit set in parallel. The road classification unit is used to process the image data using an improved lightweight convolutional neural network and output road category labels. The road elevation estimation unit is used to process the lidar point cloud data using a Kalman filter-enhanced random sampling consensus algorithm, fit the local road plane, and calculate the elevation value of the road ahead.

4. The method according to claim 3, characterized in that, The improved lightweight convolutional neural network adopts a hybrid attention-guided one-dimensional feature extraction mechanism. For multi-scale features, it first fuses channel and spatial attention to generate a unified mask, then performs attention-guided spatial max pooling to retain highly discriminative local information, and combines a multi-head classifier to output road category labels.

5. The method according to claim 3, characterized in that, The Kalman filter-enhanced random sample consensus algorithm specifically processes the following: A state-space model with road plane equation coefficients as state vectors is constructed. Point cloud data between time frames are fused using a Kalman filter. The plane parameter estimate of the previous time step is used as a priori, and the plane parameters obtained by fitting the current frame through the RANSAC algorithm are used as observations. Recursive state updates are performed to obtain local road plane estimates. Road elevation values ​​are then analytically calculated based on the local road plane estimates.

6. The method according to claim 1, characterized in that, The hybrid synchronization strategy reinforcement learning control module achieves dynamic coupling learning in the following way: Set up a round buffer and a replay buffer to store the complete interaction trajectory and single-step transition samples across rounds, respectively; During the same policy learning phase, based on the complete trajectory in the round buffer, the weighted multi-round expected reward is calculated, and the policy network and value network are updated based on the policy gradient method. During the heterogeneous policy learning phase, historical data is periodically sampled from the replay buffer to update the Q-function ensemble and policy network. The Q-network ensemble is used to optimize the most conservative Q-value, and a Q-distribution network is introduced to impose distribution constraints to mitigate overestimation. Through a synchronization mechanism, the randomly selected Q-network estimate is aligned with the multi-round expected reward calculated in the same policy phase during each training cycle, so as to achieve synchronization of the learning objectives of the two learning paradigms.

7. The method according to claim 6, characterized in that, The weighted multi-round expected return The calculation formula is: ， in, , , These represent the k-step rewards calculated at time step t for the current round and the two previous historical rounds, respectively. , , These are learnable weight parameters.

8. The method according to claim 1, characterized in that, In step S4, the action space is adaptively divided according to the road category label, specifically as follows: The original action output by the policy network is compressed by the tanh function and then multiplied by a scaling factor related to the current road category to obtain the final control force acting on the actuator. For rough roads, the scaling factor is larger, and the actuator outputs a greater force. For smooth roads, the scaling factor is smaller, and the actuator outputs a smaller force.

9. The method according to claim 1, characterized in that, The hybrid synchronization strategy reinforcement learning control module uses a soft normalization method to design the reward function, specifically including: The state variables are normalized using the extreme values ​​of the passive suspension response under typical excitation as the normalization benchmark; the reward function includes an overflow penalty imposed on states that exceed the response range of the passive system, a smooth negative reward imposed on key performance indicators, and a sparse positive reward imposed on acceleration and suspension deflection below a certain threshold.

Images